{"what":"The Swift host (apps/ClefFlash, the clef-flash CLI, Release) on the fixture (213 runs) and the held-out set (40 runs): the decoder as AOT .aimodelc and as the .aimodel specialized by the Swift runtime (JIT), fp16 and int8mix, against the author's fp32 oracle and against the Python reference of the same assets; the renderer against CPython; the tiles and patches against Pillow and the oracle's pixel_values; the tower rows against the Python runtime; one negative control.","sources":[{"file":"results/swift_gate_fp16_aot.json","bytes":676557,"sha256":"ec24b3214cd79e405543d828406ceeb77bc65166b693643fe87128cc405e1632"},{"file":"results/swift_gate_fp16_jit.json","bytes":667824,"sha256":"554db5ee6a4e36dd35d88e0312856cfef09ba252785b0eaa32d13398df73041d"},{"file":"results/swift_gate_int8mix_aot.json","bytes":609034,"sha256":"6a567fdc656b819d9bb3ca689a9b2850d0d4dab0656edf9a3e7a4c6aa9d3452d"},{"file":"results/swift_gate_int8mix_jit.json","bytes":611826,"sha256":"07f8111054bae709f1a8fe90dec514465d24bc6e6e7f37457f1758cf91407732"},{"file":"results/swift_render_check.json","bytes":866,"sha256":"409ba3402e8e42c011b7e544f460d740b6eb1ad68df275d42f4d77a826c89552"},{"file":"results/swift_pixels.json","bytes":17964,"sha256":"8ea5f1904a231ea429e7ea5305b302ba55dd34f9a5b6b73f2a47926ffc379a65"},{"file":"results/swift_towers.json","bytes":8361,"sha256":"10552f4e819409a9f82ce55761986c2eb9cbb90f79001cc9d50613d09c4a012c"},{"file":"results/swift_negative_control.json","bytes":1915,"sha256":"6bd0c0b435bdc0f047745336a5f56bbdc25ecc582f862fac63204371f3211b20"}],"trimmed":{"removed":["<asset>.passes.<set>.questions (per-question probabilities; every run row keeps max / mean |dp| vs the oracle, the bit-equality flags vs the Python reference and the response checks)","fp16_jit / int8mix_jit .passes.<set>.runs (the per-run rows of the JIT passes; each JIT asset keeps its pass summaries, per_arm, vs_python and same_as = the run-by-run comparison with the AOT passes)"],"serialization":"compact json (no indentation)","full_copies":"the source files above, in the lane directory $ZOO_WORK_ROOT/_clefflash/ (not in the repository)"},"fp16_aot":{"schema":"clef-flash-swift-gate/1","asset":"fp16_aot","asset_note":"AOT h16c .aimodelc, SpecializationOptions.default","gate":"Swift host (apps/ClefFlash, clef-flash CLI) vs the author's fp32 oracle, and vs the Python reference of the same assets","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"source":"readout_gate.BAR (unchanged)"},"package":{"path":"<zoo>/apps/ClefFlash","files_sha256":{"Package.swift":"17bc993477be7cadb01e46a4a05d21e4b899ee29d360a8005dbf28662b7bda98","README.md":"80b4fde7bb164ff636896e9ad4487d646dc4c0928310dd2d682759c9709d9ffc","Sources/ClefFlash/ClefDecider.swift":"e52a1d3b3cd4d11e64146824b069bdb32d3c017d2db6cbff54e3e47d9135b2b9","Sources/ClefFlash/Decoder.swift":"2ee502b96977cd3ea14a2668d791d073d7978870ad0f3ba43bc86527554835cd","Sources/ClefFlash/Head.swift":"671fc5adbe34427471f41e97dc9b87c6bea489ed93d42653208e23b697394c89","Sources/ClefFlash/ImagePreprocess.swift":"6f18f0592fb6accf63f6aee11dd8106812db0297ac44aba38ab6b9a253364ebc","Sources/ClefFlash/JSONValue.swift":"bb184d942a66934e8cea895c0ddb05e988ba00cec9b509eafe5ce5d7571039bf","Sources/ClefFlash/PromptBuilder.swift":"7d5fa87d532d199022339698f8e14bce8a721e4aefda0dfdc5ce5272ff047110","Sources/ClefFlash/Renderer.swift":"963e0ff25b217b19f7be86f6bf6da5c2c2781649d52a13c069f7be1841a8ce49","Sources/ClefFlash/Support.swift":"a96745ace5cdde3d7e55001bf134e12c9ff067e7f18e68aeb7011fb44729e85b","Sources/ClefFlash/SystemOne.swift":"12e38a5829f073d4515bbeb64dff4a1992b190a0632a5db1cfb52ec28326d771","Sources/ClefFlash/VisionTower.swift":"4f1b3e8d1d97bb18366460f9548862e15f33d6b8ed999456ad8eb5cfd5de456d","Sources/clef-flash/main.swift":"50384ac0836684fa4795e6d2c2dd088038255c98185b6ed6ed66ec9e4a4c433d","_time_mac.sh":"b4f36c9b5b9e9ccd30f26c1b7db94f197d84f1b48b4252cc0af61f40c4326726"},"resolved":{"eventsource":"1.5.1","swift-asn1":"1.7.3","swift-collections":"1.7.1","swift-crypto":"4.5.2","swift-huggingface":"0.12.0","swift-jinja":"2.5.1","swift-transformers":"1.3.4","yyjson":"0.12.0"},"binary":{"path":"<lane>/swift/.build/release/clef-flash","sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","bytes":8600088}},"generated_at":"2026-10-03T06:36:02+00:00","passes":{"fixture":{"set":"fixture","fixture_json":"<lane>/swift/gate/fp16_aot/fixture.json","label":"r6 fp16_aot fixture","assets":{"asset":"aot","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","decoder":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"decoder_options":"SpecializationOptions.default","head_options":"SpecializationOptions.default","tower_options":"SpecializationOptions.default"},"binary_sha256":"not in the pass (built before the CLI recorded it; L/swift/binary_sha256.txt line 1)","load_first":{"wall_s":5.399609583,"tokenizer_s":0.6297090000000001,"decoder_s":{"model":0.000874125,"main":2.3419547080000003},"head_s":0.30011625000000003,"tower_s":{"g256":1.074157,"g448":1.043874917}},"reset_check":{"run":"own_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":0.9515738330000001},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":31798317056,"entries":["0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":31798317056,"entries":["0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":31798317056,"entries":["0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"photo_01","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{"photo_01/native":"40 x 30 = 1,200 image rows > the graph's 1,024-row buffer (as in readout_gate.py)"},"summary":{"runs":213,"expected_runs":213,"questions":401,"questions_non_near_tie":397,"argmax_equal_non_near_tie":397,"near_tie_questions":4,"argmax_equal_near_tie":4,"max_abs_dp":0.01221764087677002,"mean_of_run_mean_abs_dp":0.0005437990627295013,"max_abs_dlogit_vs_oracle":0.10230028629302979,"worst_run":{"id":"img_07","arm":"g256","max_abs_dp":0.01221764087677002},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":213,"spans_equal_oracle":213,"softmax_recomputed_bit_equal_runs":213,"response_recomputed_equal_runs":213,"near_ties":[{"id":"img_01","arm":"g256","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.490953266620636,0.4879729747772217,0.02107377164065838],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_01","arm":"native","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.4912445545196533,0.487699419260025,0.02105601318180561],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_06","arm":"g448","question_id":"in_red_zone","oracle_top2_margin":0.017518073320388794,"argmax_equal":true,"probs":[0.5098239779472351,0.4901759922504425],"probs_oracle":[0.5087590217590332,0.4912409484386444]},{"id":"photo_01","arm":"g256","question_id":"kinds","oracle_top2_margin":0.00908571481704712,"argmax_equal":true,"probs":[0.02351612038910389,0.11496458202600479,0.38072216510772705,0.391661673784256,0.0891355499625206],"probs_oracle":[0.023699797689914703,0.11542528122663498,0.3812732696533203,0.39035898447036743,0.08924264460802078]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":172,"questions":255,"argmax_equal_non_near_tie":"255/255","max_abs_dp":0.010642945766448975,"mean_of_run_mean_abs_dp":0.0005945426708829175},"g256":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"48/48","max_abs_dp":0.01221764087677002,"mean_of_run_mean_abs_dp":0.0005778034727207828},"g448":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"49/49","max_abs_dp":0.004833877086639404,"mean_of_run_mean_abs_dp":0.00024134174184370493},"native":{"runs":13,"questions":46,"argmax_equal_non_near_tie":"45/45","max_abs_dp":0.0013368427753448486,"mean_of_run_mean_abs_dp":0.00016152522812454956}},"vs_python":{"all":{"runs":213,"hidden_bit_equal_runs":185,"hidden_max_abs_diff_where_unequal":3.671875,"max_abs_dp_vs_python_author_head":0.0009737610816955566,"logits_bit_equal_runs_python_head_graph":185,"max_abs_dp_vs_python_head_graph":0.000973820686340332,"logits_max_abs_diff_python_head_graph":0.006946682929992676,"response_equal_decide_py":"12/12","response_max_abs_diff_decide_py":0.0},"by_arm":{"text":{"runs":172,"hidden_bit_equal_runs":172,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07,"logits_bit_equal_runs_python_head_graph":172,"max_abs_dp_vs_python_head_graph":0.0,"logits_max_abs_diff_python_head_graph":0.0,"response_equal_decide_py":"7/7","response_max_abs_diff_decide_py":0.0},"g256":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":0.78125,"max_abs_dp_vs_python_author_head":0.0009737610816955566,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.000973820686340332,"logits_max_abs_diff_python_head_graph":0.006946682929992676,"response_equal_decide_py":"2/2","response_max_abs_diff_decide_py":0.0},"g448":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.671875,"max_abs_dp_vs_python_author_head":0.0006627440452575684,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.0006627440452575684,"logits_max_abs_diff_python_head_graph":0.004836320877075195,"response_equal_decide_py":"3/3","response_max_abs_diff_decide_py":0.0},"native":{"runs":13,"hidden_bit_equal_runs":13,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07,"logits_bit_equal_runs_python_head_graph":13,"max_abs_dp_vs_python_head_graph":0.0,"logits_max_abs_diff_python_head_graph":0.0}},"python_readout":"readout_fp16_pf64_full.json","python_readout_sha256":"89e16d622d644dd6293b0a443991d819bce8128a0fc464b38d5cbad8040c97e4","python_head_graph":"head_h2_fixture.json","python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"86d0a881b0f8e515374c1164077694d7f8d6df681e7832f51d26c98101579dde","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"568d03d87f4ef9cb4c572e626cd8934baf3e5b76969182e3d119807783e1611c","stats.json":"c5f7a6d342fb7c087b365e99a845108c4f3d723c8b3c61fa4af5f7c136808d53"}},"per_invocation":[{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"e8eed8dbde699d6f04be32d92ed312b29278eb4696c3dd329af08a4adea989a2","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"de5ae8726d5c4a438930352e7a96da6d5e17924e767a83987b3f4258ba0ae505","stats.json":"b2a8f1bc52cbeec0a1d30e1dca5b7c691cd75249135af7ac556bcdeb49676050"}}},{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"86d0a881b0f8e515374c1164077694d7f8d6df681e7832f51d26c98101579dde","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"568d03d87f4ef9cb4c572e626cd8934baf3e5b76969182e3d119807783e1611c","stats.json":"c5f7a6d342fb7c087b365e99a845108c4f3d723c8b3c61fa4af5f7c136808d53"}}}],"aimodelc_same_bytes_every_invocation":false}},"runs":[{"id":"img_01","arm":"g256","source":"own_image","tokens":430,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0014986991882324219,"mean_abs_dp":0.0004523161121390083,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.34375,"vs_python_author_head_max_abs_dp":0.0002912282943725586,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0013484954833984375,"vs_python_head_graph_max_abs_dp":0.000291287899017334,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"img_01","arm":"g448","source":"own_image","tokens":562,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.000726163387298584,"mean_abs_dp":0.00035997598686001516,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2578125,"vs_python_author_head_max_abs_dp":0.00021693110466003418,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0022068023681640625,"vs_python_head_graph_max_abs_dp":0.00021693110466003418},{"id":"img_01","arm":"native","source":"own_image","tokens":430,"calls":7,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.0013368427753448486,"mean_abs_dp":0.00045962995764884084,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_02","arm":"g256","source":"own_image","tokens":466,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0006942898035049438,"mean_abs_dp":0.00017730874771421605,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.265625,"vs_python_author_head_max_abs_dp":0.00013791024684906006,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.00084686279296875,"vs_python_head_graph_max_abs_dp":0.0001379251480102539},{"id":"img_02","arm":"g448","source":"own_image","tokens":598,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0006797909736633301,"mean_abs_dp":0.00018154296346686104,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3125,"vs_python_author_head_max_abs_dp":0.00012692809104919434,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.002406597137451172,"vs_python_head_graph_max_abs_dp":0.0001268833875656128},{"id":"img_02","arm":"native","source":"own_image","tokens":702,"calls":11,"bucket":"t1024","image_rows_from":"file","max_abs_dp":0.00045239925384521484,"mean_abs_dp":0.00013014733452688563,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_03","arm":"g256","source":"own_image","tokens":495,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0004349946975708008,"mean_abs_dp":0.00012732967734336854,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.296875,"vs_python_author_head_max_abs_dp":0.00033098459243774414,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.001417398452758789,"vs_python_head_graph_max_abs_dp":0.00033104419708251953},{"id":"img_03","arm":"g448","source":"own_image","tokens":627,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0007433891296386719,"mean_abs_dp":0.00017764437943696977,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":1.40625,"vs_python_author_head_max_abs_dp":5.745887756347656e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0025146007537841797,"vs_python_head_graph_max_abs_dp":5.7578086853027344e-05},{"id":"img_03","arm":"native","source":"own_image","tokens":501,"calls":8,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.0004870891571044922,"mean_abs_dp":0.00012751587977011998,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_04","arm":"g256","source":"own_image","tokens":558,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00024241209030151367,"mean_abs_dp":9.231337656577429e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.25,"vs_python_author_head_max_abs_dp":2.872943878173828e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0019236207008361816,"vs_python_head_graph_max_abs_dp":2.872943878173828e-05},{"id":"img_04","arm":"g448","source":"own_image","tokens":690,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00042134523391723633,"mean_abs_dp":0.00011400155102213224,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.4501953125,"vs_python_author_head_max_abs_dp":4.9173831939697266e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0018197298049926758,"vs_python_head_graph_max_abs_dp":4.9173831939697266e-05,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"img_04","arm":"native","source":"own_image","tokens":1262,"calls":20,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.0005682706832885742,"mean_abs_dp":0.0001020798459649086,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_05","arm":"g256","source":"own_image","tokens":464,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.00017082691192626953,"mean_abs_dp":6.258987228978764e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.78125,"vs_python_author_head_max_abs_dp":0.00021147727966308594,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0018479228019714355,"vs_python_head_graph_max_abs_dp":0.00021135807037353516},{"id":"img_05","arm":"g448","source":"own_image","tokens":596,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0001569986343383789,"mean_abs_dp":7.734329185702585e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.4296875,"vs_python_author_head_max_abs_dp":5.072355270385742e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.00176239013671875,"vs_python_head_graph_max_abs_dp":5.0649046897888184e-05},{"id":"img_05","arm":"native","source":"own_image","tokens":464,"calls":8,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.00013369321823120117,"mean_abs_dp":5.030869082971053e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_06","arm":"g256","source":"own_image","tokens":465,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0014993548393249512,"mean_abs_dp":0.0007205829955637455,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.203125,"vs_python_author_head_max_abs_dp":0.00019116699695587158,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0015083551406860352,"vs_python_head_graph_max_abs_dp":0.0001911371946334839},{"id":"img_06","arm":"g448","source":"own_image","tokens":597,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0021254420280456543,"mean_abs_dp":0.000827670842409134,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.5009765625,"vs_python_author_head_max_abs_dp":0.0006627440452575684,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0026556849479675293,"vs_python_head_graph_max_abs_dp":0.0006627440452575684,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"img_06","arm":"native","source":"own_image","tokens":471,"calls":8,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.0012745857238769531,"mean_abs_dp":0.0005537913180887699,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_07","arm":"g256","source":"own_image","tokens":426,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.01221764087677002,"mean_abs_dp":0.0046619860455393795,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2734375,"vs_python_author_head_max_abs_dp":0.0009737610816955566,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.006946682929992676,"vs_python_head_graph_max_abs_dp":0.000973820686340332},{"id":"img_07","arm":"g448","source":"own_image","tokens":558,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0008584856986999512,"mean_abs_dp":0.00019936440512537956,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.4375,"vs_python_author_head_max_abs_dp":0.00038444995880126953,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.003848552703857422,"vs_python_head_graph_max_abs_dp":0.00038444995880126953},{"id":"img_07","arm":"native","source":"own_image","tokens":662,"calls":11,"bucket":"t1024","image_rows_from":"file","max_abs_dp":0.00027978047728538513,"mean_abs_dp":7.085399702191352e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_08","arm":"g256","source":"own_image","tokens":522,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0015458464622497559,"mean_abs_dp":0.0003375595302454063,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.14453125,"vs_python_author_head_max_abs_dp":0.0002536177635192871,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0051784515380859375,"vs_python_head_graph_max_abs_dp":0.0002536773681640625,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"img_08","arm":"g448","source":"own_image","tokens":654,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.000510096549987793,"mean_abs_dp":0.00015859964436718395,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.9140625,"vs_python_author_head_max_abs_dp":0.00030535459518432617,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.004836320877075195,"vs_python_head_graph_max_abs_dp":0.00030535459518432617},{"id":"img_08","arm":"native","source":"own_image","tokens":1226,"calls":20,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.001103043556213379,"mean_abs_dp":0.0002860772157354014,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_09","arm":"g256","source":"own_image","tokens":460,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0019435286521911621,"mean_abs_dp":0.0003614833747798746,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.609375,"vs_python_author_head_max_abs_dp":0.0001316070556640625,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0048370361328125,"vs_python_head_graph_max_abs_dp":0.0001315474510192871},{"id":"img_09","arm":"g448","source":"own_image","tokens":592,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0005063265562057495,"mean_abs_dp":0.00010103664614937522,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.296875,"vs_python_author_head_max_abs_dp":5.27501106262207e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0019974708557128906,"vs_python_head_graph_max_abs_dp":5.27501106262207e-05},{"id":"img_09","arm":"native","source":"own_image","tokens":696,"calls":11,"bucket":"t1024","image_rows_from":"file","max_abs_dp":0.00023049116134643555,"mean_abs_dp":7.358371195468035e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"img_10","arm":"g256","source":"own_image","tokens":405,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0002390444278717041,"mean_abs_dp":9.415102087789112e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.21875,"vs_python_author_head_max_abs_dp":0.00011952221393585205,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.005314469337463379,"vs_python_head_graph_max_abs_dp":0.00011947751045227051},{"id":"img_10","arm":"g448","source":"own_image","tokens":537,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00017021596431732178,"mean_abs_dp":8.815863273209996e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.26953125,"vs_python_author_head_max_abs_dp":7.778406143188477e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0027191638946533203,"vs_python_head_graph_max_abs_dp":7.778406143188477e-05},{"id":"img_10","arm":"native","source":"own_image","tokens":405,"calls":7,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.00017770379781723022,"mean_abs_dp":8.096452802419662e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j01","arm":"text","source":"own_json","tokens":476,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0012674182653427124,"mean_abs_dp":0.0003848218824714422,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"own_j02","arm":"text","source":"own_json","tokens":475,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0010565221309661865,"mean_abs_dp":0.00023182185197418386,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j03","arm":"text","source":"own_json","tokens":585,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0018704235553741455,"mean_abs_dp":0.00033807421901396343,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j04","arm":"text","source":"own_json","tokens":569,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.005557417869567871,"mean_abs_dp":0.0009066607869629349,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j05","arm":"text","source":"own_json","tokens":539,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0033428072929382324,"mean_abs_dp":0.0005667551265408596,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j06","arm":"text","source":"own_json","tokens":616,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0012636184692382812,"mean_abs_dp":0.00020470659193747183,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j07","arm":"text","source":"own_json","tokens":595,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0012389570474624634,"mean_abs_dp":0.0002826069830916822,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j08","arm":"text","source":"own_json","tokens":473,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0010389983654022217,"mean_abs_dp":0.0003438945859670639,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":8.940696716308594e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"own_j09","arm":"text","source":"own_json","tokens":573,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.00140303373336792,"mean_abs_dp":0.0005376609818389019,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j10","arm":"text","source":"own_json","tokens":413,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0008875727653503418,"mean_abs_dp":0.0003986296836625446,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j11","arm":"text","source":"own_json","tokens":463,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0007089376449584961,"mean_abs_dp":0.0002444388810545206,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_j12","arm":"text","source":"own_json","tokens":562,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0012025237083435059,"mean_abs_dp":0.00039697237885915314,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"own_t01","arm":"text","source":"own_text","tokens":436,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0015636682510375977,"mean_abs_dp":0.00033182622864842416,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"own_t02","arm":"text","source":"own_text","tokens":599,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0012312531471252441,"mean_abs_dp":0.00022528563360018388,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t03","arm":"text","source":"own_text","tokens":463,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00822708010673523,"mean_abs_dp":0.003257949171321733,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"own_t04","arm":"text","source":"own_text","tokens":493,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003347955644130707,"mean_abs_dp":0.0001286946406418627,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t05","arm":"text","source":"own_text","tokens":553,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.00026407837867736816,"mean_abs_dp":6.600426366695992e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t06","arm":"text","source":"own_text","tokens":411,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0005132555961608887,"mean_abs_dp":0.00015263316325015493,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.470348358154297e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t07","arm":"text","source":"own_text","tokens":438,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.006536126136779785,"mean_abs_dp":0.0013508718499603372,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t08","arm":"text","source":"own_text","tokens":604,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.00021904706954956055,"mean_abs_dp":3.754263485057486e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t09","arm":"text","source":"own_text","tokens":582,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0014589130878448486,"mean_abs_dp":0.0002433915506117046,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t10","arm":"text","source":"own_text","tokens":396,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004936456680297852,"mean_abs_dp":0.00017888809088617563,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t11","arm":"text","source":"own_text","tokens":497,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0008802413940429688,"mean_abs_dp":0.00022363324056972158,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t12","arm":"text","source":"own_text","tokens":967,"calls":16,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0012277215719223022,"mean_abs_dp":0.0002732836994631537,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t13","arm":"text","source":"own_text","tokens":370,"calls":6,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0008168071508407593,"mean_abs_dp":0.00022820383310317993,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t14","arm":"text","source":"own_text_long","tokens":2389,"calls":38,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.001191943883895874,"mean_abs_dp":0.00022992951174577076,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"own_t15","arm":"text","source":"own_text_long","tokens":2603,"calls":41,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.0005500614643096924,"mean_abs_dp":7.559955966743556e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"own_t16","arm":"text","source":"own_text_long","tokens":2494,"calls":39,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.0008679032325744629,"mean_abs_dp":0.00011320964825388633,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"photo_01","arm":"g256","source":"cc0_photo","tokens":556,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0046607255935668945,"mean_abs_dp":0.0007580828387290239,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.28125,"vs_python_author_head_max_abs_dp":0.0009064674377441406,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0032852888107299805,"vs_python_head_graph_max_abs_dp":0.0009063482284545898},{"id":"photo_01","arm":"g448","source":"cc0_photo","tokens":688,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.004833877086639404,"mean_abs_dp":0.0008730270928936079,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2109375,"vs_python_author_head_max_abs_dp":0.00037151575088500977,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0018110275268554688,"vs_python_head_graph_max_abs_dp":0.00037154555320739746},{"id":"photo_02","arm":"g256","source":"cc0_photo","tokens":499,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0010362863540649414,"mean_abs_dp":0.0001712812540622858,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.15625,"vs_python_author_head_max_abs_dp":0.0003215670585632324,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0025136470794677734,"vs_python_head_graph_max_abs_dp":0.0003215670585632324},{"id":"photo_02","arm":"g448","source":"cc0_photo","tokens":631,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0005533695220947266,"mean_abs_dp":0.0001101468403178912,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.546875,"vs_python_author_head_max_abs_dp":4.45246696472168e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0008645057678222656,"vs_python_head_graph_max_abs_dp":4.4442713260650635e-05,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"photo_02","arm":"native","source":"cc0_photo","tokens":1035,"calls":17,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.00014072656631469727,"mean_abs_dp":4.566159959022816e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.0978193283081055e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"photo_03","arm":"g256","source":"cc0_photo","tokens":505,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":9.28640365600586e-05,"mean_abs_dp":4.061288200318813e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.625,"vs_python_author_head_max_abs_dp":4.1604042053222656e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.001316070556640625,"vs_python_head_graph_max_abs_dp":4.1604042053222656e-05},{"id":"photo_03","arm":"g448","source":"cc0_photo","tokens":637,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0002511143684387207,"mean_abs_dp":7.599413705368836e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2890625,"vs_python_author_head_max_abs_dp":4.3332576751708984e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0008782148361206055,"vs_python_head_graph_max_abs_dp":4.32133674621582e-05},{"id":"photo_03","arm":"native","source":"cc0_photo","tokens":1041,"calls":17,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.00018852949142456055,"mean_abs_dp":8.01204393307368e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.332993507385254e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"photo_04","arm":"g256","source":"cc0_photo","tokens":502,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0001361370086669922,"mean_abs_dp":3.165089023800997e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.25,"vs_python_author_head_max_abs_dp":3.1206756830215454e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0007486343383789062,"vs_python_head_graph_max_abs_dp":3.118440508842468e-05},{"id":"photo_04","arm":"g448","source":"cc0_photo","tokens":634,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00014495849609375,"mean_abs_dp":3.427797212050511e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":3.671875,"vs_python_author_head_max_abs_dp":3.534555435180664e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0010292232036590576,"vs_python_head_graph_max_abs_dp":3.534555435180664e-05},{"id":"photo_04","arm":"native","source":"cc0_photo","tokens":1038,"calls":17,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.00010752677917480469,"mean_abs_dp":3.9093447132752495e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_005dcf6d2c0f21d64796","arm":"text","source":"semif_authored144","tokens":209,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00026667118072509766,"mean_abs_dp":0.0001777807871500651,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_00bb4cc043f7d2b288d4","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.697008550167084e-05,"mean_abs_dp":5.127800007661184e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_02f2b95a5c5a4777a45c","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.6693957149982452e-05,"mean_abs_dp":1.1131477852662405e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_03d919fd0e9b3a7cd1e6","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00016380101442337036,"mean_abs_dp":0.00010921061038970947,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_07ee70cfe4b01330ce89","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000603020191192627,"mean_abs_dp":0.0004019998013973236,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_0aa1447680e4f25862bb","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.007522463798522949,"mean_abs_dp":0.005014980832735698,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_0aa24a6701ac9c56c4a0","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004139542579650879,"mean_abs_dp":0.0002759825438261032,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_0b43ea8e24e74d621f1b","arm":"text","source":"semif_authored144","tokens":177,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00012189149856567383,"mean_abs_dp":8.125106493631999e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_0ebd299739302d8b057b","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.001270301640033722,"mean_abs_dp":0.0008468752106030782,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_1105577da4c8dad4609d","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.010464847087860107,"mean_abs_dp":0.0069765374064445496,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.470348358154297e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_19d3d8d0bf773e88be7b","arm":"text","source":"semif_authored144","tokens":172,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.5139579772949219e-05,"mean_abs_dp":1.0049901902675629e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_19f37d5a97034bd7d04f","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00015783309936523438,"mean_abs_dp":0.00010521213213602702,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_1a81ead69e93e5d0c86a","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004739850759506226,"mean_abs_dp":0.003159940242767334,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_1df851a202994a07f123","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00012897327542304993,"mean_abs_dp":8.599087595939636e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_2175379f4c69e6207626","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00016289949417114258,"mean_abs_dp":0.00010856365164120992,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_21ca7a8b970e36b8e20f","arm":"text","source":"semif_authored144","tokens":214,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00819706916809082,"mean_abs_dp":0.005464700361092885,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_22ec57822db50ec121cf","arm":"text","source":"semif_authored144","tokens":171,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.0371208190917969e-05,"mean_abs_dp":6.886509557565053e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.0244548320770264e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_25172cd6ddc49ab0ca70","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.298798739910126e-05,"mean_abs_dp":2.8628235061963398e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_2608370d01783735c3ea","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00013047456741333008,"mean_abs_dp":8.70165725549062e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_2708193212d8a4a523c7","arm":"text","source":"semif_authored144","tokens":207,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00259321928024292,"mean_abs_dp":0.0017287979523340862,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_2bf540d515a51755b25c","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.267692565917969e-05,"mean_abs_dp":2.8438555697600048e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_31eb4533d2c89959ec46","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00019150972366333008,"mean_abs_dp":0.0001277091602484385,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_3226202ba1ada2570e9f","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004112720489501953,"mean_abs_dp":0.0002742074429988861,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_32f36cd0fbbfb3604586","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.9596110582351685e-05,"mean_abs_dp":2.6363258560498554e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_33d6e5da58f0fee2490d","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.252527236938477e-05,"mean_abs_dp":4.166302581628164e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_36273dd4777876a26338","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.611513137817383e-05,"mean_abs_dp":5.0747767090797424e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_36c71bb649b7d5eccb1f","arm":"text","source":"semif_authored144","tokens":174,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":8.18353146314621e-06,"mean_abs_dp":5.4426491260528564e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_37cebc3792e98049439f","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00024712085723876953,"mean_abs_dp":0.00016476276020208994,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_386e10b1c1ce906473a4","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.99758529663086e-05,"mean_abs_dp":4.6675714353720345e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_3b0958cabc339005cf66","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00018423795700073242,"mean_abs_dp":0.0001227923979361852,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_3dd9c3ac98f8ddf7bdd3","arm":"text","source":"semif_authored144","tokens":179,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00016170740127563477,"mean_abs_dp":0.0001077918956677119,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.421438694000244e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_3e75f2b84e8e08e3d3f2","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00022258982062339783,"mean_abs_dp":0.00014840935667355856,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_3f783eec5c90812479bc","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000321805477142334,"mean_abs_dp":0.0002145376056432724,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4089c199b8b796ec3ed8","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.002208597958087921,"mean_abs_dp":0.0014723911881446838,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_42384af9ae56beb37ff5","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000572502613067627,"mean_abs_dp":0.00038165133446455,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_42c54fa2ad45743e24d8","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.7621164917945862e-05,"mean_abs_dp":1.8410074214140575e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4444dad25e438f128673","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00011444091796875,"mean_abs_dp":7.630946735541026e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_44574c44e243ee1b2f52","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":9.989738464355469e-05,"mean_abs_dp":6.660384436448415e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_44bae0c8a1615a096c43","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.01865690946579e-06,"mean_abs_dp":2.6775524020195007e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4533dd91267b4d815690","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.94529914855957e-05,"mean_abs_dp":5.295065542062124e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_46b7029b9a704138b77a","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000141143798828125,"mean_abs_dp":9.410517911116283e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_46d6a0731bd288c370f0","arm":"text","source":"semif_authored144","tokens":182,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00014454126358032227,"mean_abs_dp":9.636208415031433e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4aa83af95bb69e90bc2d","arm":"text","source":"semif_authored144","tokens":212,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000591576099395752,"mean_abs_dp":0.00039437909921010333,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4aee37a6a155747766aa","arm":"text","source":"semif_authored144","tokens":206,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00016642361879348755,"mean_abs_dp":0.00011095901330312093,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4b3d794c004566610181","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00038182735443115234,"mean_abs_dp":0.000254549706975619,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4d7a003342db5b0a17e3","arm":"text","source":"semif_authored144","tokens":187,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":8.696317672729492e-05,"mean_abs_dp":5.79847643772761e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.166496753692627e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_4da6aedc1139224eeff2","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.824731826782227e-05,"mean_abs_dp":4.5536396404107414e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.6566128730773926e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_51ceab3ce622c7c759f1","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.918741226196289e-05,"mean_abs_dp":3.944026927153269e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_533d4423d311de82b27d","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.5914440155029297e-05,"mean_abs_dp":1.0584481060504913e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.3969838619232178e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_56c419b12cd37cbe790c","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00016129016876220703,"mean_abs_dp":0.00010754416386286418,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_579beed184ffc1104aa9","arm":"text","source":"semif_authored144","tokens":182,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.00543212890625e-05,"mean_abs_dp":2.6752861837546032e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_587ced261c6092cd1325","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.378225326538086e-05,"mean_abs_dp":1.5862596531709034e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_599b3092b6fd18312d23","arm":"text","source":"semif_authored144","tokens":187,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003300309181213379,"mean_abs_dp":0.00021999267240365347,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_5a37d1dfcad20810a76a","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0011305809020996094,"mean_abs_dp":0.0007536895573139191,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_5d31b4bf2d0fc193f53b","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003349184989929199,"mean_abs_dp":0.00022328210373719534,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_6149a17bc154f9c5b4a0","arm":"text","source":"semif_authored144","tokens":200,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0005390644073486328,"mean_abs_dp":0.00035939862330754596,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.0978193283081055e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_61ac20b07a5189231e15","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0007973909378051758,"mean_abs_dp":0.0005316144476334254,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_64d63f86e328aff1481c","arm":"text","source":"semif_authored144","tokens":210,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":9.417533874511719e-06,"mean_abs_dp":6.287048260370891e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_674ca4e9346216255466","arm":"text","source":"semif_authored144","tokens":176,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00017136335372924805,"mean_abs_dp":0.00011424471934636433,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_695dd61195ca025c4f4c","arm":"text","source":"semif_authored144","tokens":179,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.1696090698242188e-05,"mean_abs_dp":1.4468406637509664e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_6d64a8459f64eac7f4cc","arm":"text","source":"semif_authored144","tokens":207,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.133690476417542e-05,"mean_abs_dp":4.0868918100992836e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_6db17cc22bd656d78558","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00018066167831420898,"mean_abs_dp":0.00012047030031681061,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_712f6ffb763eef7c9a12","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.519390106201172e-05,"mean_abs_dp":3.679189831018448e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.7939677238464355e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_75b470586834df8c568d","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0006379485130310059,"mean_abs_dp":0.0004253070801496506,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_76c7a79b1ff2363a60c5","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.662441253662109e-06,"mean_abs_dp":3.8106615344683328e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_774f7f9245f83c796d44","arm":"text","source":"semif_authored144","tokens":180,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0002422332763671875,"mean_abs_dp":0.0001614931970834732,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_796d5c0da6eaadae6a3f","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":8.821487426757812e-06,"mean_abs_dp":5.854604144891103e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.6566128730773926e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_7ada97b6f99f4788e5c7","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.5703182220458984e-05,"mean_abs_dp":2.3816712200641632e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_7ed75d854d99b3a81916","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0023435354232788086,"mean_abs_dp":0.0015623581906159718,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4156103134155273e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_823bf736b92e9d8fa66d","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0005645155906677246,"mean_abs_dp":0.0003763437271118164,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_82c2cd97c160ad27951b","arm":"text","source":"semif_authored144","tokens":171,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0002600476145744324,"mean_abs_dp":0.00017338618636131287,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_834f3b3e10cc61b33d25","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.70280647277832e-05,"mean_abs_dp":3.131603201230367e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_843185706adff0c60f8b","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0011990368366241455,"mean_abs_dp":0.0007993355393409729,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.470348358154297e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_86a15618ba4cb90a307e","arm":"text","source":"semif_authored144","tokens":179,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.002375304698944092,"mean_abs_dp":0.0015835165977478027,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_8a4c3de28be95c105ca1","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.003479182720184326,"mean_abs_dp":0.0023194526632626853,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.705522537231445e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_8c992e17e4fb54b044f5","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.21540641784668e-05,"mean_abs_dp":3.478489816188812e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_8e8c3804a3c15ebb31e3","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0006719157099723816,"mean_abs_dp":0.00044794318576653797,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_90494c76078ffc3c717a","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00011616945266723633,"mean_abs_dp":7.748045027256012e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_92ec2ffa3dd46ada1335","arm":"text","source":"semif_authored144","tokens":204,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004864335060119629,"mean_abs_dp":0.0003243274986743927,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_942ec65dac0bfb9874f9","arm":"text","source":"semif_authored144","tokens":177,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000499427318572998,"mean_abs_dp":0.00033293478190898895,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_9473603b44af63beab79","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001342296600341797,"mean_abs_dp":8.947402238845825e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_960a4bcd27363e6db4b9","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0010715723037719727,"mean_abs_dp":0.0007143914699554443,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_97a35499bd8717f3e786","arm":"text","source":"semif_authored144","tokens":203,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001862049102783203,"mean_abs_dp":0.00012409687042236328,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.3527612686157227e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_97d2ed2591fc7638806c","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0065749287605285645,"mean_abs_dp":0.004383298257986705,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_980e76145c925747220c","arm":"text","source":"semif_authored144","tokens":209,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0009320229291915894,"mean_abs_dp":0.0006213237841924032,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_9ac5cc82c4126c91174a","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.571676254272461e-05,"mean_abs_dp":3.0496468146642048e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_a0128e551d095f3449f8","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00013750791549682617,"mean_abs_dp":9.167380630970001e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_a02fd582d2881cc462b2","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.492273390293121e-05,"mean_abs_dp":2.327871819337209e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.6763806343078613e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_a0a39ace7e5337d038a7","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.1801719665527344e-05,"mean_abs_dp":7.828697562217712e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_a22dab8ba073ff6cdb8d","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.1317973732948303e-06,"mean_abs_dp":1.4603137969970703e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_a3f18f3a63d45345942b","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.4199485778808594e-05,"mean_abs_dp":1.615658402442932e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.6763806343078613e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"semif_a9230edfc6ec9e524af0","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.283687591552734e-05,"mean_abs_dp":4.85380490620931e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_acacd8b46acd25cb954b","arm":"text","source":"semif_authored144","tokens":206,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.2590160369873047e-05,"mean_abs_dp":1.5050172805786133e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b45c63fb92729af27f05","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0017579197883605957,"mean_abs_dp":0.001171929140885671,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b49ecda75086d8c68d2a","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0010090731084346771,"mean_abs_dp":0.0006727017462253571,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b531dd603bf080dab820","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00018590688705444336,"mean_abs_dp":0.00012391060590744019,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b5e6788d95b8395d2888","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.2479248046875e-05,"mean_abs_dp":4.8308943708737694e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b7c4e62d07986286fce0","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0053250789642333984,"mean_abs_dp":0.003550062576929728,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b8091be929b1e489fcc4","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004099607467651367,"mean_abs_dp":0.00027332579096158344,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b863998d4be2721a5dfd","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00035303831100463867,"mean_abs_dp":0.00023537129163742065,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b8c5b9b285cbdddc9f8d","arm":"text","source":"semif_authored144","tokens":187,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.363059997558594e-05,"mean_abs_dp":2.908427268266678e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_b8e2750307df9761e716","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00014644861221313477,"mean_abs_dp":9.761626521746318e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_be2dd67bc733c1213bef","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00030612945556640625,"mean_abs_dp":0.00020407761136690775,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.3527612686157227e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_be866e1dbec5d42d9423","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.29425048828125e-05,"mean_abs_dp":4.196291168530782e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_c100e84585149a04be49","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.606081008911133e-05,"mean_abs_dp":2.40374356508255e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_c4e9bd463b6fde5be743","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":8.058547973632812e-05,"mean_abs_dp":5.371216684579849e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_c6a411e7f2bbdf377173","arm":"text","source":"semif_authored144","tokens":207,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00021141767501831055,"mean_abs_dp":0.00014090662201245627,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_c89b86e3f55123bfa506","arm":"text","source":"semif_authored144","tokens":182,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.7507543563842773e-05,"mean_abs_dp":1.8344571193059284e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_caf773b23934fa7362a1","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00014907121658325195,"mean_abs_dp":9.935783843199412e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_cdf7f428857e1b841cb3","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.4424324035644531e-05,"mean_abs_dp":9.599141776561737e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_cdfa9eed51fafcf09b72","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0017167329788208008,"mean_abs_dp":0.001144486169020335,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.470348358154297e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_d0e36c4cb10a29b3191c","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0018828511238098145,"mean_abs_dp":0.0012552415331204732,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_d167e51bb1f499099645","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004296362400054932,"mean_abs_dp":0.0028642316659291587,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_d41072dc35d8cd14c056","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.3048133850097656e-05,"mean_abs_dp":3.534431258837382e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_d51b05bfb9d6d7f09331","arm":"text","source":"semif_authored144","tokens":203,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.007256358861923218,"mean_abs_dp":0.004837561398744583,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_d6de731cca52359b1c87","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.003719151020050049,"mean_abs_dp":0.002479424079259237,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_d7e84e4245a29817da1f","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.277727127075195e-05,"mean_abs_dp":4.85380490620931e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.6566128730773926e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_d880e65fef5e2993fdd0","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0023258328437805176,"mean_abs_dp":0.001550545295079549,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_def9fe59d262357a673a","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003795623779296875,"mean_abs_dp":0.00025300805767377216,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_e0c140e9222d1bce5647","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.3890286684036255e-05,"mean_abs_dp":1.5913198391596477e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_e1d610bd14e3d16c09b9","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.337860107421875e-05,"mean_abs_dp":2.228282392024994e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_e332aa927d0c2ed044aa","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.9166618585586548e-06,"mean_abs_dp":1.2485931317011516e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.421438694000244e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_e6155431ab7b0f8170d4","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.044120967388153e-05,"mean_abs_dp":2.0289793610572815e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_ea7ec66a5e8eac886548","arm":"text","source":"semif_authored144","tokens":192,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004500746726989746,"mean_abs_dp":0.00030002494653066,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_eac5121a4a1bf391a08c","arm":"text","source":"semif_authored144","tokens":173,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.2040138244628906e-05,"mean_abs_dp":8.001302679379782e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_eafc22c8c40df3932a8e","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0013892054557800293,"mean_abs_dp":0.0009261531134446462,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_ebd74aa05324a1556288","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001131892204284668,"mean_abs_dp":7.550356288750966e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_ebea4681e3dcc7d92ca2","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0014224648475646973,"mean_abs_dp":0.0009483347336451212,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_ec94ef909f7918695ab3","arm":"text","source":"semif_authored144","tokens":180,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.264448165893555e-05,"mean_abs_dp":4.176671306292216e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_edb708bd0fe5fd67d6ee","arm":"text","source":"semif_authored144","tokens":210,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00035506486892700195,"mean_abs_dp":0.00023671860496203104,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_eeae283841ce75dd25bf","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.111734986305237e-05,"mean_abs_dp":2.074489990870158e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_ef3e0f8e657bf59f8a29","arm":"text","source":"semif_authored144","tokens":200,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.007280170917510986,"mean_abs_dp":0.004853422443072001,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_f2b4ec4930322fe45118","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0006383657455444336,"mean_abs_dp":0.00042557281752427417,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.421438694000244e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_f3d13cf184f916677def","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00034171342849731445,"mean_abs_dp":0.00022781267762184143,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_f40beba9088c8db8bbd6","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0006181001663208008,"mean_abs_dp":0.00041206181049346924,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_f41ae66b3c09630c7a42","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00012940168380737305,"mean_abs_dp":8.626437435547511e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_f46f392ef9e9e9df564b","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":9.071826934814453e-05,"mean_abs_dp":6.042793393135071e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_f6d68126b43f8c9ed183","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.3990869522094727e-06,"mean_abs_dp":1.5813857316970825e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_f756e1cb04e11766341e","arm":"text","source":"semif_authored144","tokens":206,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.010642945766448975,"mean_abs_dp":0.0070952574412028,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_fb73a85d386cf59b0c51","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004906117916107178,"mean_abs_dp":0.003270761420329412,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_fda0ca94652c6d739ecf","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.827976226806641e-06,"mean_abs_dp":3.2372772693634033e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_fdb9fc214aea2710763a","arm":"text","source":"semif_authored144","tokens":168,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0009981989860534668,"mean_abs_dp":0.0006654411554336548,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_fdd902cc6e65fad7cb4f","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0018469691276550293,"mean_abs_dp":0.0012313500046730042,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"semif_ffee861474a9665a093a","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0021082162857055664,"mean_abs_dp":0.0014054377873738606,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0}]},"heldout":{"set":"heldout","fixture_json":"<lane>/swift/gate/fp16_aot/heldout.json","label":"r6 fp16_aot heldout","assets":{"asset":"aot","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","decoder":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"decoder_options":"SpecializationOptions.default","head_options":"SpecializationOptions.default","tower_options":"SpecializationOptions.default"},"binary_sha256":"not in the pass (built before the CLI recorded it; L/swift/binary_sha256.txt line 1)","load_first":{"wall_s":3.537564625,"tokenizer_s":0.6501442500000001,"decoder_s":{"model":0.0007774580000000001,"main":2.503053},"head_s":0.063948083,"tower_s":{"g256":0.169799042,"g448":0.140230875}},"reset_check":{"run":"ho_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":1.625140084},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":31798317056,"entries":["0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":31798317056,"entries":["0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":31798317056,"entries":["0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"ho_i01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i02","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i03","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i04","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i05","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i06","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i07","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i08","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p02","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{},"summary":{"runs":40,"expected_runs":40,"questions":188,"questions_non_near_tie":186,"argmax_equal_non_near_tie":186,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.008229672908782959,"mean_of_run_mean_abs_dp":0.0003716305157203246,"max_abs_dlogit_vs_oracle":0.08628550171852112,"worst_run":{"id":"ho_j06","arm":"text","max_abs_dp":0.008229672908782959},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":40,"spans_equal_oracle":40,"softmax_recomputed_bit_equal_runs":40,"response_recomputed_equal_runs":40,"near_ties":[{"id":"ho_j01","arm":"text","question_id":"late_start","oracle_top2_margin":0.0010575354099273682,"argmax_equal":true,"probs":[0.5003471374511719,0.4996528923511505],"probs_oracle":[0.5005287528038025,0.4994712173938751]},{"id":"ho_t07","arm":"text","question_id":"questions_asked","oracle_top2_margin":0.011684805154800415,"argmax_equal":true,"probs":[0.009835842996835709,0.4039638340473175,0.40109479427337646,0.18510542809963226],"probs_oracle":[0.009856987744569778,0.410186767578125,0.3985019624233246,0.18145422637462616]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":20,"questions":106,"argmax_equal_non_near_tie":"104/104","max_abs_dp":0.008229672908782959,"mean_of_run_mean_abs_dp":0.0004698964852733023},"g256":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.0034676194190979004,"mean_of_run_mean_abs_dp":0.00028443024198129835},"g448":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.007532060146331787,"mean_of_run_mean_abs_dp":0.000262298850353395}},"vs_python":{"all":{"runs":40,"hidden_bit_equal_runs":20,"hidden_max_abs_diff_where_unequal":3.21875,"max_abs_dp_vs_python_author_head":0.001549839973449707,"logits_bit_equal_runs_python_head_graph":20,"max_abs_dp_vs_python_head_graph":0.001549839973449707,"logits_max_abs_diff_python_head_graph":0.03701984882354736,"response_equal_decide_py":"12/12","response_max_abs_diff_decide_py":0.0},"by_arm":{"text":{"runs":20,"hidden_bit_equal_runs":20,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":2.384185791015625e-07,"logits_bit_equal_runs_python_head_graph":20,"max_abs_dp_vs_python_head_graph":0.0,"logits_max_abs_diff_python_head_graph":0.0,"response_equal_decide_py":"7/7","response_max_abs_diff_decide_py":0.0},"g256":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.21875,"max_abs_dp_vs_python_author_head":0.001549839973449707,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.001549839973449707,"logits_max_abs_diff_python_head_graph":0.03701984882354736,"response_equal_decide_py":"2/2","response_max_abs_diff_decide_py":0.0},"g448":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":1.765625,"max_abs_dp_vs_python_author_head":0.0009412467479705811,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.0009412765502929688,"logits_max_abs_diff_python_head_graph":0.006737470626831055,"response_equal_decide_py":"3/3","response_max_abs_diff_decide_py":0.0},"native":{"runs":0}},"python_readout":"readout_heldout_fp16_pf64.json","python_readout_sha256":"49a2553cbe8ff955e47a524c1f8b8ce1f151e9998cae7ebb141a096a52c3b5e9","python_head_graph":"head_h2_heldout.json","python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"196c089dca5446037f583f9957a5fa5eac5717324e7b49dfce7cc9b7fd49ec9e","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"2c6acb63c7baf6f6072fb2d06b15d6163e95d83c56c05e302c79c90060ffe786","stats.json":"c4eac63e4b03f3d77179ecfe949163a5b64281cca93484f54a2a9ba9c813e50c"}}}},"runs":[{"id":"ho_i01","arm":"g256","source":"heldout_image","tokens":529,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0016260743141174316,"mean_abs_dp":0.00031582778319716454,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.328125,"vs_python_author_head_max_abs_dp":0.0002478957176208496,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0026859045028686523,"vs_python_head_graph_max_abs_dp":0.0002479851245880127,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_i01","arm":"g448","source":"heldout_image","tokens":661,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.001061767339706421,"mean_abs_dp":0.00021770429940750965,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.6796875,"vs_python_author_head_max_abs_dp":0.00038042664527893066,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.002649068832397461,"vs_python_head_graph_max_abs_dp":0.00038042664527893066},{"id":"ho_i02","arm":"g256","source":"heldout_image","tokens":570,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0004642009735107422,"mean_abs_dp":0.00013435818254947662,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.328125,"vs_python_author_head_max_abs_dp":0.00017696619033813477,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.004341781139373779,"vs_python_head_graph_max_abs_dp":0.00017684698104858398},{"id":"ho_i02","arm":"g448","source":"heldout_image","tokens":702,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0008227825164794922,"mean_abs_dp":0.00018327260789062296,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":1.2421875,"vs_python_author_head_max_abs_dp":3.0219554901123047e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0012996196746826172,"vs_python_head_graph_max_abs_dp":3.0219554901123047e-05,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_i03","arm":"g256","source":"heldout_image","tokens":541,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0001658797264099121,"mean_abs_dp":2.7564761694520712e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.375,"vs_python_author_head_max_abs_dp":5.40614128112793e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.001371920108795166,"vs_python_head_graph_max_abs_dp":5.4001808166503906e-05},{"id":"ho_i03","arm":"g448","source":"heldout_image","tokens":673,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00021541118621826172,"mean_abs_dp":3.982576890848577e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.36798095703125,"vs_python_author_head_max_abs_dp":3.236532211303711e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.003226339817047119,"vs_python_head_graph_max_abs_dp":3.236532211303711e-05},{"id":"ho_i04","arm":"g256","source":"heldout_image","tokens":437,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0013992488384246826,"mean_abs_dp":0.00035658718552440406,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.140625,"vs_python_author_head_max_abs_dp":0.000467836856842041,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.002414703369140625,"vs_python_head_graph_max_abs_dp":0.0004680156707763672},{"id":"ho_i04","arm":"g448","source":"heldout_image","tokens":569,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0009886622428894043,"mean_abs_dp":0.0003801428712904453,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.23046875,"vs_python_author_head_max_abs_dp":0.0003154873847961426,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.001110076904296875,"vs_python_head_graph_max_abs_dp":0.0003154277801513672},{"id":"ho_i05","arm":"g256","source":"heldout_image","tokens":701,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0016439557075500488,"mean_abs_dp":0.0004782697097176597,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3173828125,"vs_python_author_head_max_abs_dp":0.0006074309349060059,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.010030746459960938,"vs_python_head_graph_max_abs_dp":0.0006074309349060059},{"id":"ho_i05","arm":"g448","source":"heldout_image","tokens":833,"calls":14,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0013512969017028809,"mean_abs_dp":0.00020372200136383375,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":1.765625,"vs_python_author_head_max_abs_dp":0.0004674196243286133,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.006737470626831055,"vs_python_head_graph_max_abs_dp":0.0004674196243286133,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_i06","arm":"g256","source":"heldout_image","tokens":628,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.001216113567352295,"mean_abs_dp":0.0002936354691260739,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.37890625,"vs_python_author_head_max_abs_dp":0.0002797544002532959,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.00232696533203125,"vs_python_head_graph_max_abs_dp":0.0002796351909637451},{"id":"ho_i06","arm":"g448","source":"heldout_image","tokens":760,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.007532060146331787,"mean_abs_dp":0.000947302636249285,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.265625,"vs_python_author_head_max_abs_dp":0.0009412467479705811,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.004121303558349609,"vs_python_head_graph_max_abs_dp":0.0009412765502929688},{"id":"ho_i07","arm":"g256","source":"heldout_image","tokens":535,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0006528496742248535,"mean_abs_dp":0.00023361795152036042,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":3.21875,"vs_python_author_head_max_abs_dp":0.00021451711654663086,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0030884742736816406,"vs_python_head_graph_max_abs_dp":0.00021445751190185547},{"id":"ho_i07","arm":"g448","source":"heldout_image","tokens":667,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.001863718032836914,"mean_abs_dp":0.00034272602687661466,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.33984375,"vs_python_author_head_max_abs_dp":0.00029037147760391235,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0040700435638427734,"vs_python_head_graph_max_abs_dp":0.0002903789281845093},{"id":"ho_i08","arm":"g256","source":"heldout_image","tokens":581,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0007134079933166504,"mean_abs_dp":0.0001881077666493023,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.21875,"vs_python_author_head_max_abs_dp":0.00019985437393188477,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.002516508102416992,"vs_python_head_graph_max_abs_dp":0.00019985437393188477},{"id":"ho_i08","arm":"g448","source":"heldout_image","tokens":713,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0006529465317726135,"mean_abs_dp":0.0001350393080536057,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.533203125,"vs_python_author_head_max_abs_dp":0.00015157461166381836,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0054267048835754395,"vs_python_head_graph_max_abs_dp":0.00015157461166381836},{"id":"ho_j01","arm":"text","source":"heldout_json","tokens":769,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0005374550819396973,"mean_abs_dp":0.00014550617585579556,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_j02","arm":"text","source":"heldout_json","tokens":335,"calls":6,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0011948645114898682,"mean_abs_dp":0.0004763146862387657,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_j03","arm":"text","source":"heldout_json","tokens":775,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0018700957298278809,"mean_abs_dp":0.0002910656070238666,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_j04","arm":"text","source":"heldout_json","tokens":610,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.00146406888961792,"mean_abs_dp":0.00042268486383060616,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_j05","arm":"text","source":"heldout_json","tokens":775,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0017920732498168945,"mean_abs_dp":0.00044073186376515557,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_j06","arm":"text","source":"heldout_json","tokens":733,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.008229672908782959,"mean_abs_dp":0.001465221271953649,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_j07","arm":"text","source":"heldout_json","tokens":826,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0015503168106079102,"mean_abs_dp":0.0003872389659389026,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_j08","arm":"text","source":"heldout_json","tokens":798,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0043204426765441895,"mean_abs_dp":0.001021276745531294,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_l01","arm":"text","source":"heldout_text_long","tokens":2850,"calls":45,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.0012440681457519531,"mean_abs_dp":0.0002137862518429756,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_l02","arm":"text","source":"heldout_text_long","tokens":2731,"calls":43,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.0006785988807678223,"mean_abs_dp":0.00014048570301383734,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_p01","arm":"g256","source":"heldout_photo","tokens":634,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0002702176570892334,"mean_abs_dp":8.718607326348622e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.35546875,"vs_python_author_head_max_abs_dp":8.45789909362793e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0014309883117675781,"vs_python_head_graph_max_abs_dp":8.45789909362793e-05},{"id":"ho_p01","arm":"g448","source":"heldout_photo","tokens":766,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00026954710483551025,"mean_abs_dp":0.00010205854972203573,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.546875,"vs_python_author_head_max_abs_dp":6.64442777633667e-05,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0013755559921264648,"vs_python_head_graph_max_abs_dp":6.645917892456055e-05,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_p02","arm":"g256","source":"heldout_photo","tokens":620,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0034676194190979004,"mean_abs_dp":0.0007291475365705349,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.26904296875,"vs_python_author_head_max_abs_dp":0.001549839973449707,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.03701984882354736,"vs_python_head_graph_max_abs_dp":0.001549839973449707,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_p02","arm":"g448","source":"heldout_photo","tokens":752,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00030487775802612305,"mean_abs_dp":7.119443377151209e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.578125,"vs_python_author_head_max_abs_dp":0.00018912553787231445,"logits_bit_equal_python_head_graph":false,"logits_max_abs_diff_python_head_graph":0.0034439265727996826,"vs_python_head_graph_max_abs_dp":0.00018900632858276367},{"id":"ho_t01","arm":"text","source":"heldout_text","tokens":708,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0023886561393737793,"mean_abs_dp":0.00033675205400761436,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_t02","arm":"text","source":"heldout_text","tokens":746,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0018744468688964844,"mean_abs_dp":0.0003860886790789664,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.384185791015625e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_t03","arm":"text","source":"heldout_text","tokens":842,"calls":14,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.00241011381149292,"mean_abs_dp":0.00028710811748169365,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_t04","arm":"text","source":"heldout_text","tokens":658,"calls":11,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0008854269981384277,"mean_abs_dp":0.00013302513980306685,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_t05","arm":"text","source":"heldout_text","tokens":727,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0002464801073074341,"mean_abs_dp":9.086175954767636e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_t06","arm":"text","source":"heldout_text","tokens":721,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0009756684303283691,"mean_abs_dp":0.00017992193427156,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_t07","arm":"text","source":"heldout_text","tokens":1086,"calls":17,"bucket":"t2048","image_rows_from":"none","max_abs_dp":0.006222933530807495,"mean_abs_dp":0.0007646634275543278,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0,"response_equal_decide_py":true,"response_max_abs_diff_decide_py":0.0},{"id":"ho_t08","arm":"text","source":"heldout_text","tokens":864,"calls":14,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.002567768096923828,"mean_abs_dp":0.00043065121863037346,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_t09","arm":"text","source":"heldout_text","tokens":867,"calls":14,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0032619237899780273,"mean_abs_dp":0.0009531376676426993,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0},{"id":"ho_t10","arm":"text","source":"heldout_text","tokens":800,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0036197900772094727,"mean_abs_dp":0.0008314075724532207,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07,"logits_bit_equal_python_head_graph":true,"logits_max_abs_diff_python_head_graph":0.0,"vs_python_head_graph_max_abs_dp":0.0}]}},"oracle_rows_pass":{"fixture_json":["<lane>/swift/gate/fp16_aot/fixture_rows.json","<lane>/swift/gate/fp16_aot/heldout_rows.json"],"what":"g256 / g448 runs fed the oracle's fp32 rows (the readout_gate.py inputs): the Swift decoder + head vs the Python decoder + head on identical inputs","reset_checks":[{"run":"img_01/g256","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":0.9496782920000001},{"run":"ho_i01/g256","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":1.220714291}],"fixture":{"runs":28,"hidden_bit_equal":28,"logits_bit_equal_head_graph":28,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07,"rows":[{"id":"img_01","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_01","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.341104507446289e-07},{"id":"img_02","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_02","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_03","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_03","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07},{"id":"img_04","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_04","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_05","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_05","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_06","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":8.940696716308594e-08},{"id":"img_06","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07},{"id":"img_07","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_07","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":6.705522537231445e-08},{"id":"img_08","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_08","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_09","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_09","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":4.470348358154297e-08},{"id":"img_10","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":6.332993507385254e-08},{"id":"img_10","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1175870895385742e-08},{"id":"photo_01","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_01","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_02","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_02","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_03","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_03","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_04","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":2.2351741790771484e-08},{"id":"photo_04","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07}]},"heldout":{"runs":20,"hidden_bit_equal":20,"logits_bit_equal_head_graph":20,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07,"rows":[{"id":"ho_i01","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":8.940696716308594e-08},{"id":"ho_i01","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.862645149230957e-08},{"id":"ho_i02","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i02","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.4901161193847656e-08},{"id":"ho_i03","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i03","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":2.9802322387695312e-08},{"id":"ho_i04","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07},{"id":"ho_i04","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":8.940696716308594e-08},{"id":"ho_i05","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i05","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i06","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i06","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"ho_i07","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i07","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i08","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":2.2351741790771484e-08},{"id":"ho_i08","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_p01","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":2.9802322387695312e-08},{"id":"ho_p01","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_p02","arm":"g256","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_p02","arm":"g448","hidden_bit_equal_python":true,"logits_bit_equal_python_head_graph":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07}]}},"combined":{"runs":253,"questions_non_near_tie":583,"argmax_equal_non_near_tie":583,"near_tie_questions":6,"argmax_equal_near_tie":6,"max_abs_dp":0.01221764087677002,"result":"PASS"}},"fp16_jit":{"schema":"clef-flash-swift-gate/1","asset":"fp16_jit","asset_note":"the .aimodel specialized by Swift (GPU preferred + expectFrequentReshapes)","gate":"Swift host (apps/ClefFlash, clef-flash CLI) vs the author's fp32 oracle, and vs the Python reference of the same assets","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"source":"readout_gate.BAR (unchanged)"},"package":{"path":"<zoo>/apps/ClefFlash","files_sha256":{"Package.swift":"17bc993477be7cadb01e46a4a05d21e4b899ee29d360a8005dbf28662b7bda98","README.md":"80b4fde7bb164ff636896e9ad4487d646dc4c0928310dd2d682759c9709d9ffc","Sources/ClefFlash/ClefDecider.swift":"e52a1d3b3cd4d11e64146824b069bdb32d3c017d2db6cbff54e3e47d9135b2b9","Sources/ClefFlash/Decoder.swift":"2ee502b96977cd3ea14a2668d791d073d7978870ad0f3ba43bc86527554835cd","Sources/ClefFlash/Head.swift":"671fc5adbe34427471f41e97dc9b87c6bea489ed93d42653208e23b697394c89","Sources/ClefFlash/ImagePreprocess.swift":"6f18f0592fb6accf63f6aee11dd8106812db0297ac44aba38ab6b9a253364ebc","Sources/ClefFlash/JSONValue.swift":"bb184d942a66934e8cea895c0ddb05e988ba00cec9b509eafe5ce5d7571039bf","Sources/ClefFlash/PromptBuilder.swift":"7d5fa87d532d199022339698f8e14bce8a721e4aefda0dfdc5ce5272ff047110","Sources/ClefFlash/Renderer.swift":"963e0ff25b217b19f7be86f6bf6da5c2c2781649d52a13c069f7be1841a8ce49","Sources/ClefFlash/Support.swift":"a96745ace5cdde3d7e55001bf134e12c9ff067e7f18e68aeb7011fb44729e85b","Sources/ClefFlash/SystemOne.swift":"12e38a5829f073d4515bbeb64dff4a1992b190a0632a5db1cfb52ec28326d771","Sources/ClefFlash/VisionTower.swift":"4f1b3e8d1d97bb18366460f9548862e15f33d6b8ed999456ad8eb5cfd5de456d","Sources/clef-flash/main.swift":"50384ac0836684fa4795e6d2c2dd088038255c98185b6ed6ed66ec9e4a4c433d","_time_mac.sh":"b4f36c9b5b9e9ccd30f26c1b7db94f197d84f1b48b4252cc0af61f40c4326726"},"resolved":{"eventsource":"1.5.1","swift-asn1":"1.7.3","swift-collections":"1.7.1","swift-crypto":"4.5.2","swift-huggingface":"0.12.0","swift-jinja":"2.5.1","swift-transformers":"1.3.4","yyjson":"0.12.0"},"binary":{"path":"<lane>/swift/.build/release/clef-flash","sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","bytes":8600088}},"generated_at":"2026-10-03T06:36:03+00:00","passes":{"fixture":{"set":"fixture","fixture_json":"<lane>/swift/gate/fp16_jit/fixture.json","label":"r6 fp16_jit fixture","assets":{"asset":"jit","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","decoder":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64/clef_flash_decode_fp16_pf64.aimodel","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.aimodel","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32/clef_flash_g256_vision_fp16w32.aimodel","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32/clef_flash_g448_vision_fp16w32.aimodel"},"decoder_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes true","head_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false","tower_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false"},"binary_sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","load_first":{"wall_s":56.119748791,"tokenizer_s":0.633645625,"decoder_s":{"model":47.470026167,"main":2.528869708},"head_s":0.9961851250000001,"tower_s":{"g256":2.23739425,"g448":2.242903833}},"reset_check":{"run":"own_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":0.9811459170000001},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":31798317056,"entries":["0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":63632965632,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":63632965632,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"photo_01","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{"photo_01/native":"40 x 30 = 1,200 image rows > the graph's 1,024-row buffer (as in readout_gate.py)"},"summary":{"runs":213,"expected_runs":213,"questions":401,"questions_non_near_tie":397,"argmax_equal_non_near_tie":397,"near_tie_questions":4,"argmax_equal_near_tie":4,"max_abs_dp":0.01221764087677002,"mean_of_run_mean_abs_dp":0.0005437990627295013,"max_abs_dlogit_vs_oracle":0.10230028629302979,"worst_run":{"id":"img_07","arm":"g256","max_abs_dp":0.01221764087677002},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":213,"spans_equal_oracle":213,"softmax_recomputed_bit_equal_runs":213,"response_recomputed_equal_runs":213,"near_ties":[{"id":"img_01","arm":"g256","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.490953266620636,0.4879729747772217,0.02107377164065838],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_01","arm":"native","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.4912445545196533,0.487699419260025,0.02105601318180561],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_06","arm":"g448","question_id":"in_red_zone","oracle_top2_margin":0.017518073320388794,"argmax_equal":true,"probs":[0.5098239779472351,0.4901759922504425],"probs_oracle":[0.5087590217590332,0.4912409484386444]},{"id":"photo_01","arm":"g256","question_id":"kinds","oracle_top2_margin":0.00908571481704712,"argmax_equal":true,"probs":[0.02351612038910389,0.11496458202600479,0.38072216510772705,0.391661673784256,0.0891355499625206],"probs_oracle":[0.023699797689914703,0.11542528122663498,0.3812732696533203,0.39035898447036743,0.08924264460802078]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":172,"questions":255,"argmax_equal_non_near_tie":"255/255","max_abs_dp":0.010642945766448975,"mean_of_run_mean_abs_dp":0.0005945426708829175},"g256":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"48/48","max_abs_dp":0.01221764087677002,"mean_of_run_mean_abs_dp":0.0005778034727207828},"g448":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"49/49","max_abs_dp":0.004833877086639404,"mean_of_run_mean_abs_dp":0.00024134174184370493},"native":{"runs":13,"questions":46,"argmax_equal_non_near_tie":"45/45","max_abs_dp":0.0013368427753448486,"mean_of_run_mean_abs_dp":0.00016152522812454956}},"vs_python":{"all":{"runs":213,"hidden_bit_equal_runs":185,"hidden_max_abs_diff_where_unequal":3.671875,"max_abs_dp_vs_python_author_head":0.0009737610816955566,"logits_bit_equal_runs_python_head_graph":185,"max_abs_dp_vs_python_head_graph":0.000973820686340332,"logits_max_abs_diff_python_head_graph":0.006946682929992676,"response_equal_decide_py":"12/12","response_max_abs_diff_decide_py":0.0},"by_arm":{"text":{"runs":172,"hidden_bit_equal_runs":172,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07,"logits_bit_equal_runs_python_head_graph":172,"max_abs_dp_vs_python_head_graph":0.0,"logits_max_abs_diff_python_head_graph":0.0,"response_equal_decide_py":"7/7","response_max_abs_diff_decide_py":0.0},"g256":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":0.78125,"max_abs_dp_vs_python_author_head":0.0009737610816955566,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.000973820686340332,"logits_max_abs_diff_python_head_graph":0.006946682929992676,"response_equal_decide_py":"2/2","response_max_abs_diff_decide_py":0.0},"g448":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.671875,"max_abs_dp_vs_python_author_head":0.0006627440452575684,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.0006627440452575684,"logits_max_abs_diff_python_head_graph":0.004836320877075195,"response_equal_decide_py":"3/3","response_max_abs_diff_decide_py":0.0},"native":{"runs":13,"hidden_bit_equal_runs":13,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07,"logits_bit_equal_runs_python_head_graph":13,"max_abs_dp_vs_python_head_graph":0.0,"logits_max_abs_diff_python_head_graph":0.0}},"python_readout":"readout_fp16_pf64_full.json","python_readout_sha256":"89e16d622d644dd6293b0a443991d819bce8128a0fc464b38d5cbad8040c97e4","python_head_graph":"head_h2_fixture.json","python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"86d0a881b0f8e515374c1164077694d7f8d6df681e7832f51d26c98101579dde","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"568d03d87f4ef9cb4c572e626cd8934baf3e5b76969182e3d119807783e1611c","stats.json":"c5f7a6d342fb7c087b365e99a845108c4f3d723c8b3c61fa4af5f7c136808d53"}},"per_invocation":[{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"e8eed8dbde699d6f04be32d92ed312b29278eb4696c3dd329af08a4adea989a2","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"de5ae8726d5c4a438930352e7a96da6d5e17924e767a83987b3f4258ba0ae505","stats.json":"b2a8f1bc52cbeec0a1d30e1dca5b7c691cd75249135af7ac556bcdeb49676050"}}},{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"86d0a881b0f8e515374c1164077694d7f8d6df681e7832f51d26c98101579dde","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"568d03d87f4ef9cb4c572e626cd8934baf3e5b76969182e3d119807783e1611c","stats.json":"c5f7a6d342fb7c087b365e99a845108c4f3d723c8b3c61fa4af5f7c136808d53"}}}],"aimodelc_same_bytes_every_invocation":false}}},"heldout":{"set":"heldout","fixture_json":"<lane>/swift/gate/fp16_jit/heldout.json","label":"r6 fp16_jit heldout","assets":{"asset":"jit","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","decoder":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64/clef_flash_decode_fp16_pf64.aimodel","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.aimodel","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32/clef_flash_g256_vision_fp16w32.aimodel","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32/clef_flash_g448_vision_fp16w32.aimodel"},"decoder_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes true","head_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false","tower_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false"},"binary_sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","load_first":{"wall_s":4.470520166,"tokenizer_s":0.673972542,"decoder_s":{"model":0.0008,"main":2.4888165840000003},"head_s":0.065659125,"tower_s":{"g256":1.071377458,"g448":0.16008770800000002}},"reset_check":{"run":"ho_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":1.634340291},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":63632965632,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":63632965632,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":63632965632,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"ho_i01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i02","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i03","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i04","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i05","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i06","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i07","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i08","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p02","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{},"summary":{"runs":40,"expected_runs":40,"questions":188,"questions_non_near_tie":186,"argmax_equal_non_near_tie":186,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.008229672908782959,"mean_of_run_mean_abs_dp":0.0003716305157203246,"max_abs_dlogit_vs_oracle":0.08628550171852112,"worst_run":{"id":"ho_j06","arm":"text","max_abs_dp":0.008229672908782959},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":40,"spans_equal_oracle":40,"softmax_recomputed_bit_equal_runs":40,"response_recomputed_equal_runs":40,"near_ties":[{"id":"ho_j01","arm":"text","question_id":"late_start","oracle_top2_margin":0.0010575354099273682,"argmax_equal":true,"probs":[0.5003471374511719,0.4996528923511505],"probs_oracle":[0.5005287528038025,0.4994712173938751]},{"id":"ho_t07","arm":"text","question_id":"questions_asked","oracle_top2_margin":0.011684805154800415,"argmax_equal":true,"probs":[0.009835842996835709,0.4039638340473175,0.40109479427337646,0.18510542809963226],"probs_oracle":[0.009856987744569778,0.410186767578125,0.3985019624233246,0.18145422637462616]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":20,"questions":106,"argmax_equal_non_near_tie":"104/104","max_abs_dp":0.008229672908782959,"mean_of_run_mean_abs_dp":0.0004698964852733023},"g256":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.0034676194190979004,"mean_of_run_mean_abs_dp":0.00028443024198129835},"g448":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.007532060146331787,"mean_of_run_mean_abs_dp":0.000262298850353395}},"vs_python":{"all":{"runs":40,"hidden_bit_equal_runs":20,"hidden_max_abs_diff_where_unequal":3.21875,"max_abs_dp_vs_python_author_head":0.001549839973449707,"logits_bit_equal_runs_python_head_graph":20,"max_abs_dp_vs_python_head_graph":0.001549839973449707,"logits_max_abs_diff_python_head_graph":0.03701984882354736,"response_equal_decide_py":"12/12","response_max_abs_diff_decide_py":0.0},"by_arm":{"text":{"runs":20,"hidden_bit_equal_runs":20,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":2.384185791015625e-07,"logits_bit_equal_runs_python_head_graph":20,"max_abs_dp_vs_python_head_graph":0.0,"logits_max_abs_diff_python_head_graph":0.0,"response_equal_decide_py":"7/7","response_max_abs_diff_decide_py":0.0},"g256":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.21875,"max_abs_dp_vs_python_author_head":0.001549839973449707,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.001549839973449707,"logits_max_abs_diff_python_head_graph":0.03701984882354736,"response_equal_decide_py":"2/2","response_max_abs_diff_decide_py":0.0},"g448":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":1.765625,"max_abs_dp_vs_python_author_head":0.0009412467479705811,"logits_bit_equal_runs_python_head_graph":0,"max_abs_dp_vs_python_head_graph":0.0009412765502929688,"logits_max_abs_diff_python_head_graph":0.006737470626831055,"response_equal_decide_py":"3/3","response_max_abs_diff_decide_py":0.0},"native":{"runs":0}},"python_readout":"readout_heldout_fp16_pf64.json","python_readout_sha256":"49a2553cbe8ff955e47a524c1f8b8ce1f151e9998cae7ebb141a096a52c3b5e9","python_head_graph":"head_h2_heldout.json","python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"196c089dca5446037f583f9957a5fa5eac5717324e7b49dfce7cc9b7fd49ec9e","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"2c6acb63c7baf6f6072fb2d06b15d6163e95d83c56c05e302c79c90060ffe786","stats.json":"c4eac63e4b03f3d77179ecfe949163a5b64281cca93484f54a2a9ba9c813e50c"}}}}}},"same_as":{"other_passes":["<lane>/swift/gate/fp16_aot/fixture.json","<lane>/swift/gate/fp16_aot/heldout.json"],"runs_compared":253,"hidden_bit_equal":253,"logits_bit_equal":253,"image_rows_bit_equal":"61/61","responses_equal":253},"combined":{"runs":253,"questions_non_near_tie":583,"argmax_equal_non_near_tie":583,"near_tie_questions":6,"argmax_equal_near_tie":6,"max_abs_dp":0.01221764087677002,"result":"PASS"}},"int8mix_aot":{"schema":"clef-flash-swift-gate/1","asset":"int8mix_aot","asset_note":"AOT h16c .aimodelc, SpecializationOptions.default","gate":"Swift host (apps/ClefFlash, clef-flash CLI) vs the author's fp32 oracle, and vs the Python reference of the same assets","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"source":"readout_gate.BAR (unchanged)"},"package":{"path":"<zoo>/apps/ClefFlash","files_sha256":{"Package.swift":"17bc993477be7cadb01e46a4a05d21e4b899ee29d360a8005dbf28662b7bda98","README.md":"80b4fde7bb164ff636896e9ad4487d646dc4c0928310dd2d682759c9709d9ffc","Sources/ClefFlash/ClefDecider.swift":"e52a1d3b3cd4d11e64146824b069bdb32d3c017d2db6cbff54e3e47d9135b2b9","Sources/ClefFlash/Decoder.swift":"2ee502b96977cd3ea14a2668d791d073d7978870ad0f3ba43bc86527554835cd","Sources/ClefFlash/Head.swift":"671fc5adbe34427471f41e97dc9b87c6bea489ed93d42653208e23b697394c89","Sources/ClefFlash/ImagePreprocess.swift":"6f18f0592fb6accf63f6aee11dd8106812db0297ac44aba38ab6b9a253364ebc","Sources/ClefFlash/JSONValue.swift":"bb184d942a66934e8cea895c0ddb05e988ba00cec9b509eafe5ce5d7571039bf","Sources/ClefFlash/PromptBuilder.swift":"7d5fa87d532d199022339698f8e14bce8a721e4aefda0dfdc5ce5272ff047110","Sources/ClefFlash/Renderer.swift":"963e0ff25b217b19f7be86f6bf6da5c2c2781649d52a13c069f7be1841a8ce49","Sources/ClefFlash/Support.swift":"a96745ace5cdde3d7e55001bf134e12c9ff067e7f18e68aeb7011fb44729e85b","Sources/ClefFlash/SystemOne.swift":"12e38a5829f073d4515bbeb64dff4a1992b190a0632a5db1cfb52ec28326d771","Sources/ClefFlash/VisionTower.swift":"4f1b3e8d1d97bb18366460f9548862e15f33d6b8ed999456ad8eb5cfd5de456d","Sources/clef-flash/main.swift":"50384ac0836684fa4795e6d2c2dd088038255c98185b6ed6ed66ec9e4a4c433d","_time_mac.sh":"b4f36c9b5b9e9ccd30f26c1b7db94f197d84f1b48b4252cc0af61f40c4326726"},"resolved":{"eventsource":"1.5.1","swift-asn1":"1.7.3","swift-collections":"1.7.1","swift-crypto":"4.5.2","swift-huggingface":"0.12.0","swift-jinja":"2.5.1","swift-transformers":"1.3.4","yyjson":"0.12.0"},"binary":{"path":"<lane>/swift/.build/release/clef-flash","sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","bytes":8600088}},"generated_at":"2026-10-03T06:36:04+00:00","passes":{"fixture":{"set":"fixture","fixture_json":"<lane>/swift/gate/int8mix_aot/fixture.json","label":"r6 int8mix_aot fixture","assets":{"asset":"aot","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","decoder":"<lane>/exports/bundles_aotc/clef_flash_decode_int8mix_pf64.h16c.aimodelc","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"decoder_options":"SpecializationOptions.default","head_options":"SpecializationOptions.default","tower_options":"SpecializationOptions.default"},"binary_sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","load_first":{"wall_s":57.651246375,"tokenizer_s":0.652320458,"decoder_s":{"model":37.843162917,"main":16.470349875},"head_s":0.298214083,"tower_s":{"g256":1.184767083,"g448":1.193477458}},"reset_check":{"run":"own_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":0.950073292},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":63632965632,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":89311371264,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":89311371264,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"photo_01","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{"photo_01/native":"40 x 30 = 1,200 image rows > the graph's 1,024-row buffer (as in readout_gate.py)"},"summary":{"runs":213,"expected_runs":213,"questions":401,"questions_non_near_tie":397,"argmax_equal_non_near_tie":397,"near_tie_questions":4,"argmax_equal_near_tie":4,"max_abs_dp":0.015300750732421875,"mean_of_run_mean_abs_dp":0.0008912478670570327,"max_abs_dlogit_vs_oracle":0.2769968509674072,"worst_run":{"id":"semif_1105577da4c8dad4609d","arm":"text","max_abs_dp":0.015300750732421875},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":213,"spans_equal_oracle":213,"softmax_recomputed_bit_equal_runs":213,"response_recomputed_equal_runs":213,"near_ties":[{"id":"img_01","arm":"g256","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.4943249225616455,0.4846669137477875,0.021008146926760674],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_01","arm":"native","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.4945833384990692,0.48441118001937866,0.02100559137761593],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_06","arm":"g448","question_id":"in_red_zone","oracle_top2_margin":0.017518073320388794,"argmax_equal":true,"probs":[0.5129877328872681,0.48701226711273193],"probs_oracle":[0.5087590217590332,0.4912409484386444]},{"id":"photo_01","arm":"g256","question_id":"kinds","oracle_top2_margin":0.00908571481704712,"argmax_equal":true,"probs":[0.023665063083171844,0.11539263278245926,0.38060200214385986,0.39052292704582214,0.08981741219758987],"probs_oracle":[0.023699797689914703,0.11542528122663498,0.3812732696533203,0.39035898447036743,0.08924264460802078]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":172,"questions":255,"argmax_equal_non_near_tie":"255/255","max_abs_dp":0.015300750732421875,"mean_of_run_mean_abs_dp":0.00097288144361876},"g256":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"48/48","max_abs_dp":0.013058662414550781,"mean_of_run_mean_abs_dp":0.0007292509376840674},"g448":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"49/49","max_abs_dp":0.005241453647613525,"mean_of_run_mean_abs_dp":0.0004977952082557852},"native":{"runs":13,"questions":46,"argmax_equal_non_near_tie":"45/45","max_abs_dp":0.004015207290649414,"mean_of_run_mean_abs_dp":0.00040934933365871425}},"vs_python":{"all":{"runs":213,"hidden_bit_equal_runs":185,"hidden_max_abs_diff_where_unequal":3.625,"max_abs_dp_vs_python_author_head":0.0009214878082275391},"by_arm":{"text":{"runs":172,"hidden_bit_equal_runs":172,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07},"g256":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":0.796875,"max_abs_dp_vs_python_author_head":0.0009214878082275391},"g448":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.625,"max_abs_dp_vs_python_author_head":0.00046178698539733887},"native":{"runs":13,"hidden_bit_equal_runs":13,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07}},"python_readout":"readout_int8mix_pf64.json","python_readout_sha256":"e7eb22c1affea851f67158594029b7ff7a4369bcc45a55421dcb55c8ed5e6b2f","python_head_graph":null,"python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","name":"clef_flash_decode_int8mix_pf64","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":155,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the untied lm_head table stays on the host)"},"prefill_chunk":64,"metadata_sha256":"1eafaedf47981c83c82bf4f609615e2a33f28093b8fab3d89f86dec0e520fc36","main_mlirb":{"bytes":11826520241,"sha256":"2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_int8mix_pf64.h16c.aimodelc","bytes":25678382366,"tree_sha256":"e80f419a8a9af23390bd540270510413234361ef5f164659878f393f0b12681a","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"fc51b9aeb51a86c9fdb4061c6a6911def64e74a44008547d5e6e706d7423e4fe","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"5ac972e3a1bf9e3d91452395b2d3f874a3f2ce285694f7b50298522261226553","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f022d2d941944adfcfe6351d5ee94fc72b618eac18bd99ddb3935dc72ec6676f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"a4aef1c29efc3b7193da1c26b06495284c01311d7b4b12fa47e2b14a02ee847b","main-h16c.mlirb":"9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","main.hash":"3e20a00e80a801ac947f24ee223029c8688932ebb85e1c53369e5e42bed4ef82","metadata.json":"0d1604a74a754f6d673e475d48ce70591ab407fd83cf5a7a379055a56fd17c11","stats.json":"0c48677700237e6e276aff023c19d2ca080c23ab0b7b8631d3b959c83015c0e5"}}}},"runs":[{"id":"img_01","arm":"g256","source":"own_image","tokens":430,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.003756791353225708,"mean_abs_dp":0.0011831499128179116,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3515625,"vs_python_author_head_max_abs_dp":0.00025835633277893066},{"id":"img_01","arm":"g448","source":"own_image","tokens":562,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0028904974460601807,"mean_abs_dp":0.0008761306208643047,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2421875,"vs_python_author_head_max_abs_dp":0.00046178698539733887},{"id":"img_01","arm":"native","source":"own_image","tokens":430,"calls":7,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.004015207290649414,"mean_abs_dp":0.0012185954573479566,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"img_02","arm":"g256","source":"own_image","tokens":466,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0015123933553695679,"mean_abs_dp":0.00035844548520716756,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.25,"vs_python_author_head_max_abs_dp":0.00013719499111175537},{"id":"img_02","arm":"g448","source":"own_image","tokens":598,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0010725557804107666,"mean_abs_dp":0.00023630837147886103,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.28125,"vs_python_author_head_max_abs_dp":0.00021852552890777588},{"id":"img_02","arm":"native","source":"own_image","tokens":702,"calls":11,"bucket":"t1024","image_rows_from":"file","max_abs_dp":0.001192539930343628,"mean_abs_dp":0.000299544327638366,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":8.940696716308594e-08},{"id":"img_03","arm":"g256","source":"own_image","tokens":495,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0009365677833557129,"mean_abs_dp":0.0001988431749244531,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2421875,"vs_python_author_head_max_abs_dp":0.0005740523338317871},{"id":"img_03","arm":"g448","source":"own_image","tokens":627,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.000789940357208252,"mean_abs_dp":0.00019403186937173207,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":1.4375,"vs_python_author_head_max_abs_dp":0.00011849403381347656},{"id":"img_03","arm":"native","source":"own_image","tokens":501,"calls":8,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.0006908178329467773,"mean_abs_dp":0.00016887206584215164,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"img_04","arm":"g256","source":"own_image","tokens":558,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0005338191986083984,"mean_abs_dp":0.00011278682698806127,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.296875,"vs_python_author_head_max_abs_dp":0.0002231001853942871},{"id":"img_04","arm":"g448","source":"own_image","tokens":690,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00020918622612953186,"mean_abs_dp":6.780928621689479e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.416015625,"vs_python_author_head_max_abs_dp":0.00012832880020141602},{"id":"img_04","arm":"native","source":"own_image","tokens":1262,"calls":20,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.0003370046615600586,"mean_abs_dp":9.395092104872068e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"img_05","arm":"g256","source":"own_image","tokens":464,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0003121495246887207,"mean_abs_dp":0.0001477214761755683,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.71875,"vs_python_author_head_max_abs_dp":0.00013107061386108398},{"id":"img_05","arm":"g448","source":"own_image","tokens":596,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0005016326904296875,"mean_abs_dp":0.0002185808156024326,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.359375,"vs_python_author_head_max_abs_dp":0.00010502338409423828},{"id":"img_05","arm":"native","source":"own_image","tokens":464,"calls":8,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.00035387277603149414,"mean_abs_dp":0.00014006524262103167,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.166496753692627e-08},{"id":"img_06","arm":"g256","source":"own_image","tokens":465,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0009855031967163086,"mean_abs_dp":0.0005760831758379936,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3203125,"vs_python_author_head_max_abs_dp":0.0009214878082275391},{"id":"img_06","arm":"g448","source":"own_image","tokens":597,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.005241453647613525,"mean_abs_dp":0.0020744116976857186,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.515625,"vs_python_author_head_max_abs_dp":0.00024133920669555664},{"id":"img_06","arm":"native","source":"own_image","tokens":471,"calls":8,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.001186668872833252,"mean_abs_dp":0.0004922335036098957,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"img_07","arm":"g256","source":"own_image","tokens":426,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.013058662414550781,"mean_abs_dp":0.005367487948387861,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3125,"vs_python_author_head_max_abs_dp":0.0005713701248168945},{"id":"img_07","arm":"g448","source":"own_image","tokens":558,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00019419193267822266,"mean_abs_dp":6.8748090416193e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.375,"vs_python_author_head_max_abs_dp":0.0004569888114929199},{"id":"img_07","arm":"native","source":"own_image","tokens":662,"calls":11,"bucket":"t1024","image_rows_from":"file","max_abs_dp":0.0012724995613098145,"mean_abs_dp":0.00028281682170927525,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08},{"id":"img_08","arm":"g256","source":"own_image","tokens":522,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0009447485208511353,"mean_abs_dp":0.00028859591111540794,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.1875,"vs_python_author_head_max_abs_dp":0.00042894482612609863},{"id":"img_08","arm":"g448","source":"own_image","tokens":654,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0016759037971496582,"mean_abs_dp":0.0005792920876826559,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":1.0859375,"vs_python_author_head_max_abs_dp":0.0002492964267730713},{"id":"img_08","arm":"native","source":"own_image","tokens":1226,"calls":20,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.003094494342803955,"mean_abs_dp":0.0009689534615193095,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"img_09","arm":"g256","source":"own_image","tokens":460,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0018312633037567139,"mean_abs_dp":0.0003756329586560076,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.796875,"vs_python_author_head_max_abs_dp":0.00029030442237854004},{"id":"img_09","arm":"g448","source":"own_image","tokens":592,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0023223459720611572,"mean_abs_dp":0.00046756270934234965,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3203125,"vs_python_author_head_max_abs_dp":9.864568710327148e-05},{"id":"img_09","arm":"native","source":"own_image","tokens":696,"calls":11,"bucket":"t1024","image_rows_from":"file","max_abs_dp":0.0038951337337493896,"mean_abs_dp":0.0007688304070721973,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"img_10","arm":"g256","source":"own_image","tokens":405,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0017645955085754395,"mean_abs_dp":0.000514587697883447,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.390625,"vs_python_author_head_max_abs_dp":1.919269561767578e-05},{"id":"img_10","arm":"g448","source":"own_image","tokens":537,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0012652873992919922,"mean_abs_dp":0.000426865519127912,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.38671875,"vs_python_author_head_max_abs_dp":5.441904067993164e-05},{"id":"img_10","arm":"native","source":"own_image","tokens":405,"calls":7,"bucket":"t512","image_rows_from":"file","max_abs_dp":0.0017417669296264648,"mean_abs_dp":0.0005180063760942883,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08},{"id":"own_j01","arm":"text","source":"own_json","tokens":476,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004546329379081726,"mean_abs_dp":0.0010347020346671344,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.470348358154297e-08},{"id":"own_j02","arm":"text","source":"own_json","tokens":475,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0018439292907714844,"mean_abs_dp":0.0004986028440973976,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-07},{"id":"own_j03","arm":"text","source":"own_json","tokens":585,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0026117563247680664,"mean_abs_dp":0.0004952460793512208,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"own_j04","arm":"text","source":"own_json","tokens":569,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.008576691150665283,"mean_abs_dp":0.0014953523807759797,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_j05","arm":"text","source":"own_json","tokens":539,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.009159505367279053,"mean_abs_dp":0.0015178301682074865,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_j06","arm":"text","source":"own_json","tokens":616,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.003360182046890259,"mean_abs_dp":0.0005124190356582403,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_j07","arm":"text","source":"own_json","tokens":595,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.005722254514694214,"mean_abs_dp":0.001280651893466711,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_j08","arm":"text","source":"own_json","tokens":473,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0024087131023406982,"mean_abs_dp":0.001112032402306795,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"own_j09","arm":"text","source":"own_json","tokens":573,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.005972325801849365,"mean_abs_dp":0.0014703625347465277,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_j10","arm":"text","source":"own_json","tokens":413,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.003991305828094482,"mean_abs_dp":0.001217319630086422,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07},{"id":"own_j11","arm":"text","source":"own_json","tokens":463,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0011699795722961426,"mean_abs_dp":0.00044734925031661985,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08},{"id":"own_j12","arm":"text","source":"own_json","tokens":562,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.004523158073425293,"mean_abs_dp":0.00127956228187451,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.705522537231445e-08},{"id":"own_t01","arm":"text","source":"own_text","tokens":436,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000804603099822998,"mean_abs_dp":0.0002658821409568191,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"own_t02","arm":"text","source":"own_text","tokens":599,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0017867684364318848,"mean_abs_dp":0.00032589829061180353,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08},{"id":"own_t03","arm":"text","source":"own_text","tokens":463,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.01452985405921936,"mean_abs_dp":0.004331876523792744,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07},{"id":"own_t04","arm":"text","source":"own_text","tokens":493,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003673061728477478,"mean_abs_dp":0.00012245283208110115,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_t05","arm":"text","source":"own_text","tokens":553,"calls":9,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0009506344795227051,"mean_abs_dp":0.0001742606982588768,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_t06","arm":"text","source":"own_text","tokens":411,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0012700259685516357,"mean_abs_dp":0.00030819088634517457,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_t07","arm":"text","source":"own_text","tokens":438,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.010020248591899872,"mean_abs_dp":0.00283558484322081,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"own_t08","arm":"text","source":"own_text","tokens":604,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0025891661643981934,"mean_abs_dp":0.0003364879068815046,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"own_t09","arm":"text","source":"own_text","tokens":582,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.005182921886444092,"mean_abs_dp":0.0008738161996006966,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08},{"id":"own_t10","arm":"text","source":"own_text","tokens":396,"calls":7,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00011381879448890686,"mean_abs_dp":6.624788511544466e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_t11","arm":"text","source":"own_text","tokens":497,"calls":8,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.001582615077495575,"mean_abs_dp":0.0005371189591559497,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"own_t12","arm":"text","source":"own_text","tokens":967,"calls":16,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0007474422454833984,"mean_abs_dp":0.00018208880170152104,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":8.940696716308594e-08},{"id":"own_t13","arm":"text","source":"own_text","tokens":370,"calls":6,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0008774176239967346,"mean_abs_dp":0.00026554049691185355,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"own_t14","arm":"text","source":"own_text_long","tokens":2389,"calls":38,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.003975570201873779,"mean_abs_dp":0.0006773259801169236,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.043081283569336e-07},{"id":"own_t15","arm":"text","source":"own_text_long","tokens":2603,"calls":41,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.0022974908351898193,"mean_abs_dp":0.0003042274091223424,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"own_t16","arm":"text","source":"own_text_long","tokens":2494,"calls":39,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.002226531505584717,"mean_abs_dp":0.00038341101945230837,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"photo_01","arm":"g256","source":"cc0_photo","tokens":556,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.002764463424682617,"mean_abs_dp":0.0006381598650477827,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.296875,"vs_python_author_head_max_abs_dp":0.0008150041103363037},{"id":"photo_01","arm":"g448","source":"cc0_photo","tokens":688,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.004550039768218994,"mean_abs_dp":0.0011779610213125125,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.203125,"vs_python_author_head_max_abs_dp":0.0002416670322418213},{"id":"photo_02","arm":"g256","source":"cc0_photo","tokens":499,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.0013607144355773926,"mean_abs_dp":0.00030676755481041397,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.17578125,"vs_python_author_head_max_abs_dp":0.0002638697624206543},{"id":"photo_02","arm":"g448","source":"cc0_photo","tokens":631,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0014851093292236328,"mean_abs_dp":0.0002977798263040873,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.5,"vs_python_author_head_max_abs_dp":5.2988529205322266e-05},{"id":"photo_02","arm":"native","source":"cc0_photo","tokens":1035,"calls":17,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.0005776882171630859,"mean_abs_dp":0.0001554875634610653,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"photo_03","arm":"g256","source":"cc0_photo","tokens":505,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.00010745227336883545,"mean_abs_dp":5.8733237286408745e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.625,"vs_python_author_head_max_abs_dp":3.7670135498046875e-05},{"id":"photo_03","arm":"g448","source":"cc0_photo","tokens":637,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0007714852690696716,"mean_abs_dp":0.0001996902283281088,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.31640625,"vs_python_author_head_max_abs_dp":1.4662742614746094e-05},{"id":"photo_03","arm":"native","source":"cc0_photo","tokens":1041,"calls":17,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.00023953616619110107,"mean_abs_dp":0.0001004787627607584,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"photo_04","arm":"g256","source":"cc0_photo","tokens":502,"calls":8,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.00032007694244384766,"mean_abs_dp":8.25179024384572e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2470703125,"vs_python_author_head_max_abs_dp":2.3115426301956177e-05},{"id":"photo_04","arm":"g448","source":"cc0_photo","tokens":634,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0003161430358886719,"mean_abs_dp":8.396077184722974e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":3.625,"vs_python_author_head_max_abs_dp":1.9073486328125e-05},{"id":"photo_04","arm":"native","source":"cc0_photo","tokens":1038,"calls":17,"bucket":"t2048","image_rows_from":"file","max_abs_dp":0.0005574822425842285,"mean_abs_dp":0.0001137064268382696,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_005dcf6d2c0f21d64796","arm":"text","source":"semif_authored144","tokens":209,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004993081092834473,"mean_abs_dp":0.0003328944245974223,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_00bb4cc043f7d2b288d4","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001268722116947174,"mean_abs_dp":8.454111715157826e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09},{"id":"semif_02f2b95a5c5a4777a45c","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.0558319091796875e-05,"mean_abs_dp":4.0409465630849205e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09},{"id":"semif_03d919fd0e9b3a7cd1e6","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003425478935241699,"mean_abs_dp":0.00022836153705914816,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08},{"id":"semif_07ee70cfe4b01330ce89","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0015053749084472656,"mean_abs_dp":0.0010035547117392223,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08},{"id":"semif_0aa1447680e4f25862bb","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00742793083190918,"mean_abs_dp":0.004951976239681244,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_0aa24a6701ac9c56c4a0","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0009721517562866211,"mean_abs_dp":0.000648114209373792,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.6763806343078613e-08},{"id":"semif_0b43ea8e24e74d621f1b","arm":"text","source":"semif_authored144","tokens":177,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.592429518699646e-05,"mean_abs_dp":1.728348433971405e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-09},{"id":"semif_0ebd299739302d8b057b","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.005542159080505371,"mean_abs_dp":0.003694787621498108,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-07},{"id":"semif_1105577da4c8dad4609d","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.015300750732421875,"mean_abs_dp":0.010200485587120056,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_19d3d8d0bf773e88be7b","arm":"text","source":"semif_authored144","tokens":172,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.224611282348633e-05,"mean_abs_dp":2.155018349488576e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09},{"id":"semif_19f37d5a97034bd7d04f","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00034934282302856445,"mean_abs_dp":0.00023289645711580911,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_1a81ead69e93e5d0c86a","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.013865232467651367,"mean_abs_dp":0.009243508179982504,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_1df851a202994a07f123","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00011086463928222656,"mean_abs_dp":7.39172101020813e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_2175379f4c69e6207626","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.001274704933166504,"mean_abs_dp":0.0008497945964336395,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.3527612686157227e-08},{"id":"semif_21ca7a8b970e36b8e20f","arm":"text","source":"semif_authored144","tokens":214,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.001251816749572754,"mean_abs_dp":0.0008345519502957662,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_22ec57822db50ec121cf","arm":"text","source":"semif_authored144","tokens":171,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00014573335647583008,"mean_abs_dp":9.7148430844148e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.6566128730773926e-09},{"id":"semif_25172cd6ddc49ab0ca70","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":8.115917444229126e-05,"mean_abs_dp":5.410673717657725e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08},{"id":"semif_2608370d01783735c3ea","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00037366151809692383,"mean_abs_dp":0.0002491474151611328,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_2708193212d8a4a523c7","arm":"text","source":"semif_authored144","tokens":207,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.006249427795410156,"mean_abs_dp":0.004166300098101298,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_2bf540d515a51755b25c","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00011879205703735352,"mean_abs_dp":7.921271026134491e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-10},{"id":"semif_31eb4533d2c89959ec46","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00019271299242973328,"mean_abs_dp":0.0001285150647163391,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_3226202ba1ada2570e9f","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00040522217750549316,"mean_abs_dp":0.0002701506018638611,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.0978193283081055e-08},{"id":"semif_32f36cd0fbbfb3604586","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00042873620986938477,"mean_abs_dp":0.00028581110139687854,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_33d6e5da58f0fee2490d","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00015944242477416992,"mean_abs_dp":0.00010628936191399892,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_36273dd4777876a26338","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":9.292364120483398e-05,"mean_abs_dp":6.196027000745137e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09},{"id":"semif_36c71bb649b7d5eccb1f","arm":"text","source":"semif_authored144","tokens":174,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.6597251892089844e-05,"mean_abs_dp":2.4424244960149128e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09},{"id":"semif_37cebc3792e98049439f","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0005404949188232422,"mean_abs_dp":0.0003603224953015645,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08},{"id":"semif_386e10b1c1ce906473a4","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.0558319091796875e-05,"mean_abs_dp":4.038400948047638e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09},{"id":"semif_3b0958cabc339005cf66","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.884786903858185e-05,"mean_abs_dp":3.2572075724601746e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08},{"id":"semif_3dd9c3ac98f8ddf7bdd3","arm":"text","source":"semif_authored144","tokens":179,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0002791285514831543,"mean_abs_dp":0.00018607514599959055,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_3e75f2b84e8e08e3d3f2","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0013009905815124512,"mean_abs_dp":0.0008673258125782013,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_3f783eec5c90812479bc","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001771487295627594,"mean_abs_dp":0.00011810225745042165,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_4089c199b8b796ec3ed8","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0009483397006988525,"mean_abs_dp":0.0006322140494982401,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08},{"id":"semif_42384af9ae56beb37ff5","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003331303596496582,"mean_abs_dp":0.00022210118671258292,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.210719347000122e-08},{"id":"semif_42c54fa2ad45743e24d8","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00013369321823120117,"mean_abs_dp":8.914402375618617e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09},{"id":"semif_4444dad25e438f128673","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.3309683203697205e-05,"mean_abs_dp":2.2241845726966858e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-09},{"id":"semif_44574c44e243ee1b2f52","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00010335445404052734,"mean_abs_dp":6.889800230662028e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_44bae0c8a1615a096c43","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":9.298324584960938e-05,"mean_abs_dp":6.199690202871959e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-09},{"id":"semif_4533dd91267b4d815690","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00022798776626586914,"mean_abs_dp":0.00015197508037090302,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_46b7029b9a704138b77a","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":8.822232484817505e-05,"mean_abs_dp":5.882295469443003e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09},{"id":"semif_46d6a0731bd288c370f0","arm":"text","source":"semif_authored144","tokens":182,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004000663757324219,"mean_abs_dp":0.0002666773895422618,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_4aa83af95bb69e90bc2d","arm":"text","source":"semif_authored144","tokens":212,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0019207745790481567,"mean_abs_dp":0.0012805114189783733,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08},{"id":"semif_4aee37a6a155747766aa","arm":"text","source":"semif_authored144","tokens":206,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0007803440093994141,"mean_abs_dp":0.0005202392737070719,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08},{"id":"semif_4b3d794c004566610181","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0004113316535949707,"mean_abs_dp":0.0002742260694503784,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08},{"id":"semif_4d7a003342db5b0a17e3","arm":"text","source":"semif_authored144","tokens":187,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.0279159545898438e-05,"mean_abs_dp":2.020100752512614e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.3969838619232178e-08},{"id":"semif_4da6aedc1139224eeff2","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.4440308809280396e-05,"mean_abs_dp":2.2925746937592823e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-09},{"id":"semif_51ceab3ce622c7c759f1","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00022149085998535156,"mean_abs_dp":0.00014767982065677643,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_533d4423d311de82b27d","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.881620407104492e-05,"mean_abs_dp":3.25872873266538e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_56c419b12cd37cbe790c","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00038188695907592773,"mean_abs_dp":0.0002545919269323349,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_579beed184ffc1104aa9","arm":"text","source":"semif_authored144","tokens":182,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.2879412174224854e-05,"mean_abs_dp":2.1930783987045288e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.3969838619232178e-08},{"id":"semif_587ced261c6092cd1325","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.1553056538105011e-05,"mean_abs_dp":7.675960659980774e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_599b3092b6fd18312d23","arm":"text","source":"semif_authored144","tokens":187,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.335973739624023e-05,"mean_abs_dp":4.2249138156572975e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_5a37d1dfcad20810a76a","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0013142228126525879,"mean_abs_dp":0.0008761125306288401,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_5d31b4bf2d0fc193f53b","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":9.322166442871094e-05,"mean_abs_dp":6.212480366230011e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_6149a17bc154f9c5b4a0","arm":"text","source":"semif_authored144","tokens":200,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00038782134652137756,"mean_abs_dp":0.00025856370727221173,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_61ac20b07a5189231e15","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003541707992553711,"mean_abs_dp":0.00023611324528853098,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.0978193283081055e-08},{"id":"semif_64d63f86e328aff1481c","arm":"text","source":"semif_authored144","tokens":210,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00010263919830322266,"mean_abs_dp":6.844600041707356e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_674ca4e9346216255466","arm":"text","source":"semif_authored144","tokens":176,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00020182132720947266,"mean_abs_dp":0.00013453885912895203,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_695dd61195ca025c4f4c","arm":"text","source":"semif_authored144","tokens":179,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00012230873107910156,"mean_abs_dp":8.15410166978836e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-10},{"id":"semif_6d64a8459f64eac7f4cc","arm":"text","source":"semif_authored144","tokens":207,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.2040138244628906e-05,"mean_abs_dp":8.03917646408081e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-09},{"id":"semif_6db17cc22bd656d78558","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00025159120559692383,"mean_abs_dp":0.00016771691540877023,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-09},{"id":"semif_712f6ffb763eef7c9a12","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.718801498413086e-05,"mean_abs_dp":5.145898709694544e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-09},{"id":"semif_75b470586834df8c568d","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0006888508796691895,"mean_abs_dp":0.00045924633741378784,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-09},{"id":"semif_76c7a79b1ff2363a60c5","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001112818717956543,"mean_abs_dp":7.418294747670491e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_774f7f9245f83c796d44","arm":"text","source":"semif_authored144","tokens":180,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001119375228881836,"mean_abs_dp":7.46212899684906e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_796d5c0da6eaadae6a3f","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":9.840726852416992e-05,"mean_abs_dp":6.564768652121226e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.6566128730773926e-09},{"id":"semif_7ada97b6f99f4788e5c7","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.476308822631836e-05,"mean_abs_dp":2.9858822623888653e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09},{"id":"semif_7ed75d854d99b3a81916","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004393182694911957,"mean_abs_dp":0.002928774803876877,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_823bf736b92e9d8fa66d","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0006732344627380371,"mean_abs_dp":0.000448832909266154,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07},{"id":"semif_82c2cd97c160ad27951b","arm":"text","source":"semif_authored144","tokens":171,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0015044808387756348,"mean_abs_dp":0.00100300336877505,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07},{"id":"semif_834f3b3e10cc61b33d25","arm":"text","source":"semif_authored144","tokens":175,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00026911497116088867,"mean_abs_dp":0.00017943729956944784,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_843185706adff0c60f8b","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004034459590911865,"mean_abs_dp":0.0026896422108014426,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_86a15618ba4cb90a307e","arm":"text","source":"semif_authored144","tokens":179,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00644761323928833,"mean_abs_dp":0.004298376540342967,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08},{"id":"semif_8a4c3de28be95c105ca1","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.008306324481964111,"mean_abs_dp":0.005537554621696472,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-08},{"id":"semif_8c992e17e4fb54b044f5","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00028836727142333984,"mean_abs_dp":0.00019227402905623117,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_8e8c3804a3c15ebb31e3","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0032947659492492676,"mean_abs_dp":0.002196528638402621,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_90494c76078ffc3c717a","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.968846380710602e-05,"mean_abs_dp":3.978051245212555e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09},{"id":"semif_92ec2ffa3dd46ada1335","arm":"text","source":"semif_authored144","tokens":204,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000454973429441452,"mean_abs_dp":0.0003033044437567393,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.9802322387695312e-08},{"id":"semif_942ec65dac0bfb9874f9","arm":"text","source":"semif_authored144","tokens":177,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0010556578636169434,"mean_abs_dp":0.0007037793596585592,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_9473603b44af63beab79","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00029093027114868164,"mean_abs_dp":0.00019393054147561392,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_960a4bcd27363e6db4b9","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004209093749523163,"mean_abs_dp":0.002806084851423899,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_97a35499bd8717f3e786","arm":"text","source":"semif_authored144","tokens":203,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0007053017616271973,"mean_abs_dp":0.00047020986676216125,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_97d2ed2591fc7638806c","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.012604236602783203,"mean_abs_dp":0.008402852962414423,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-07},{"id":"semif_980e76145c925747220c","arm":"text","source":"semif_authored144","tokens":209,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0035846829414367676,"mean_abs_dp":0.0023898258805274963,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_9ac5cc82c4126c91174a","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001710057258605957,"mean_abs_dp":0.0001140199601650238,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08},{"id":"semif_a0128e551d095f3449f8","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":8.493661880493164e-06,"mean_abs_dp":5.661820371945699e-06,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_a02fd582d2881cc462b2","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00011533498764038086,"mean_abs_dp":7.687012354532878e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09},{"id":"semif_a0a39ace7e5337d038a7","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":1.52587890625e-05,"mean_abs_dp":1.0167869428793589e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09},{"id":"semif_a22dab8ba073ff6cdb8d","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.3424625396728516e-05,"mean_abs_dp":1.5631628533204395e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_a3f18f3a63d45345942b","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.1081180572509766e-05,"mean_abs_dp":3.4034873048464455e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_a9230edfc6ec9e524af0","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":6.126798689365387e-05,"mean_abs_dp":4.083290696144104e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09},{"id":"semif_acacd8b46acd25cb954b","arm":"text","source":"semif_authored144","tokens":206,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0005488991737365723,"mean_abs_dp":0.00036593154072761536,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08},{"id":"semif_b45c63fb92729af27f05","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0039885640144348145,"mean_abs_dp":0.0026590252916018167,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_b49ecda75086d8c68d2a","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.005889400839805603,"mean_abs_dp":0.003926253567139308,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.470348358154297e-08},{"id":"semif_b531dd603bf080dab820","arm":"text","source":"semif_authored144","tokens":199,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00029164552688598633,"mean_abs_dp":0.00019440179069836935,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_b5e6788d95b8395d2888","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00012856721878051758,"mean_abs_dp":8.56739158431689e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_b7c4e62d07986286fce0","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.009572327136993408,"mean_abs_dp":0.006381573776404063,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_b8091be929b1e489fcc4","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0011304020881652832,"mean_abs_dp":0.0007536274691422781,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_b863998d4be2721a5dfd","arm":"text","source":"semif_authored144","tokens":205,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0031720995903015137,"mean_abs_dp":0.0021147727966308594,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_b8c5b9b285cbdddc9f8d","arm":"text","source":"semif_authored144","tokens":187,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00014603137969970703,"mean_abs_dp":9.735114872455597e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08},{"id":"semif_b8e2750307df9761e716","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003604888916015625,"mean_abs_dp":0.00024030419687430063,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_be2dd67bc733c1213bef","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.000139603391289711,"mean_abs_dp":9.310804307460785e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09},{"id":"semif_be866e1dbec5d42d9423","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":7.593631744384766e-05,"mean_abs_dp":5.062172810236613e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.60770320892334e-08},{"id":"semif_c100e84585149a04be49","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0001755356788635254,"mean_abs_dp":0.00011704706897338231,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_c4e9bd463b6fde5be743","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":4.8454850912094116e-05,"mean_abs_dp":3.2300750414530434e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.1420419216156006e-08},{"id":"semif_c6a411e7f2bbdf377173","arm":"text","source":"semif_authored144","tokens":207,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0005664825439453125,"mean_abs_dp":0.0003776003917058309,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_c89b86e3f55123bfa506","arm":"text","source":"semif_authored144","tokens":182,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0002720355987548828,"mean_abs_dp":0.00018138190110524496,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_caf773b23934fa7362a1","arm":"text","source":"semif_authored144","tokens":190,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00028127431869506836,"mean_abs_dp":0.00018750565747419992,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.30385160446167e-08},{"id":"semif_cdf7f428857e1b841cb3","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.638599395751953e-05,"mean_abs_dp":3.755589326222738e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-10},{"id":"semif_cdfa9eed51fafcf09b72","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0018956661224365234,"mean_abs_dp":0.001263745129108429,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_d0e36c4cb10a29b3191c","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.009045124053955078,"mean_abs_dp":0.006030060350894928,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_d167e51bb1f499099645","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.003130674362182617,"mean_abs_dp":0.0020871410767237344,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":8.940696716308594e-08},{"id":"semif_d41072dc35d8cd14c056","arm":"text","source":"semif_authored144","tokens":195,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00016099214553833008,"mean_abs_dp":0.0001073175420363744,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09},{"id":"semif_d51b05bfb9d6d7f09331","arm":"text","source":"semif_authored144","tokens":203,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.011444807052612305,"mean_abs_dp":0.007629871368408203,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_d6de731cca52359b1c87","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0021852850914001465,"mean_abs_dp":0.0014568790793418884,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_d7e84e4245a29817da1f","arm":"text","source":"semif_authored144","tokens":184,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.5128014385700226e-05,"mean_abs_dp":1.672562211751938e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.6566128730773926e-09},{"id":"semif_d880e65fef5e2993fdd0","arm":"text","source":"semif_authored144","tokens":202,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0076326727867126465,"mean_abs_dp":0.005088478326797485,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_def9fe59d262357a673a","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0003204941749572754,"mean_abs_dp":0.00021364291508992514,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08},{"id":"semif_e0c140e9222d1bce5647","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.748867988586426e-05,"mean_abs_dp":3.832206130027771e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.862645149230957e-08},{"id":"semif_e1d610bd14e3d16c09b9","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.8133392333984375e-05,"mean_abs_dp":1.8764597674210865e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09},{"id":"semif_e332aa927d0c2ed044aa","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00023674964904785156,"mean_abs_dp":0.00015786165992418924,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.0489096641540527e-08},{"id":"semif_e6155431ab7b0f8170d4","arm":"text","source":"semif_authored144","tokens":170,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":2.8847716748714447e-05,"mean_abs_dp":1.923957218726476e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_ea7ec66a5e8eac886548","arm":"text","source":"semif_authored144","tokens":192,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0010184049606323242,"mean_abs_dp":0.0006788931787014008,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_eac5121a4a1bf391a08c","arm":"text","source":"semif_authored144","tokens":173,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.537271499633789e-05,"mean_abs_dp":3.695456931988398e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09},{"id":"semif_eafc22c8c40df3932a8e","arm":"text","source":"semif_authored144","tokens":189,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.003110647201538086,"mean_abs_dp":0.0020737648010253906,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.725290298461914e-08},{"id":"semif_ebd74aa05324a1556288","arm":"text","source":"semif_authored144","tokens":186,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":3.587640821933746e-05,"mean_abs_dp":2.38809734582901e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":0.0},{"id":"semif_ebea4681e3dcc7d92ca2","arm":"text","source":"semif_authored144","tokens":196,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00103035569190979,"mean_abs_dp":0.000686901311079661,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.705522537231445e-08},{"id":"semif_ec94ef909f7918695ab3","arm":"text","source":"semif_authored144","tokens":180,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0006815791130065918,"mean_abs_dp":0.00045434323449929553,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.587935447692871e-09},{"id":"semif_edb708bd0fe5fd67d6ee","arm":"text","source":"semif_authored144","tokens":210,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00040858983993530273,"mean_abs_dp":0.0002724093695481618,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.3527612686157227e-08},{"id":"semif_eeae283841ce75dd25bf","arm":"text","source":"semif_authored144","tokens":197,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.1915645599365234e-05,"mean_abs_dp":3.461074084043503e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":6.51925802230835e-09},{"id":"semif_ef3e0f8e657bf59f8a29","arm":"text","source":"semif_authored144","tokens":200,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0014143586158752441,"mean_abs_dp":0.0009428958098093668,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_f2b4ec4930322fe45118","arm":"text","source":"semif_authored144","tokens":193,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0005028843879699707,"mean_abs_dp":0.0003352655718723933,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":2.2351741790771484e-08},{"id":"semif_f3d13cf184f916677def","arm":"text","source":"semif_authored144","tokens":208,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0008071660995483398,"mean_abs_dp":0.0005380958318710327,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":3.3527612686157227e-08},{"id":"semif_f40beba9088c8db8bbd6","arm":"text","source":"semif_authored144","tokens":183,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0010207295417785645,"mean_abs_dp":0.0006804615259170532,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_f41ae66b3c09630c7a42","arm":"text","source":"semif_authored144","tokens":178,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00028705596923828125,"mean_abs_dp":0.00019133836030960083,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":7.450580596923828e-09},{"id":"semif_f46f392ef9e9e9df564b","arm":"text","source":"semif_authored144","tokens":188,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":5.0533562898635864e-05,"mean_abs_dp":3.372629483540853e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-08},{"id":"semif_f6d68126b43f8c9ed183","arm":"text","source":"semif_authored144","tokens":181,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00026732683181762695,"mean_abs_dp":0.00017822409669558206,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1175870895385742e-08},{"id":"semif_f756e1cb04e11766341e","arm":"text","source":"semif_authored144","tokens":206,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.009549587965011597,"mean_abs_dp":0.006366429229577382,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_fb73a85d386cf59b0c51","arm":"text","source":"semif_authored144","tokens":185,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0064220428466796875,"mean_abs_dp":0.004281389216581981,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"semif_fda0ca94652c6d739ecf","arm":"text","source":"semif_authored144","tokens":194,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.00010579824447631836,"mean_abs_dp":7.054458061854045e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":9.313225746154785e-09},{"id":"semif_fdb9fc214aea2710763a","arm":"text","source":"semif_authored144","tokens":168,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0002390444278717041,"mean_abs_dp":0.00015935922662417093,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":4.0978193283081055e-08},{"id":"semif_fdd902cc6e65fad7cb4f","arm":"text","source":"semif_authored144","tokens":201,"calls":4,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0008806586265563965,"mean_abs_dp":0.0005871423830588659,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"semif_ffee861474a9665a093a","arm":"text","source":"semif_authored144","tokens":191,"calls":3,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.0012271404266357422,"mean_abs_dp":0.0008180613319079081,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08}]},"heldout":{"set":"heldout","fixture_json":"<lane>/swift/gate/int8mix_aot/heldout.json","label":"r6 int8mix_aot heldout","assets":{"asset":"aot","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","decoder":"<lane>/exports/bundles_aotc/clef_flash_decode_int8mix_pf64.h16c.aimodelc","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"decoder_options":"SpecializationOptions.default","head_options":"SpecializationOptions.default","tower_options":"SpecializationOptions.default"},"binary_sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","load_first":{"wall_s":5.116695542,"tokenizer_s":0.688862541,"decoder_s":{"model":0.000832542,"main":1.989297083},"head_s":0.30604308300000005,"tower_s":{"g256":1.067890875,"g448":1.054691292}},"reset_check":{"run":"ho_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":1.628061958},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":89311371264,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":89311371264,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":89275039744,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"ho_i01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i02","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i03","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i04","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i05","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i06","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i07","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i08","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p02","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{},"summary":{"runs":40,"expected_runs":40,"questions":188,"questions_non_near_tie":186,"argmax_equal_non_near_tie":186,"near_tie_questions":2,"argmax_equal_near_tie":1,"max_abs_dp":0.017072200775146484,"mean_of_run_mean_abs_dp":0.0006477783313688305,"max_abs_dlogit_vs_oracle":0.13446736335754395,"worst_run":{"id":"ho_t08","arm":"text","max_abs_dp":0.017072200775146484},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":40,"spans_equal_oracle":40,"softmax_recomputed_bit_equal_runs":40,"response_recomputed_equal_runs":40,"near_ties":[{"id":"ho_j01","arm":"text","question_id":"late_start","oracle_top2_margin":0.0010575354099273682,"argmax_equal":true,"probs":[0.5021580457687378,0.4978419542312622],"probs_oracle":[0.5005287528038025,0.4994712173938751]},{"id":"ho_t07","arm":"text","question_id":"questions_asked","oracle_top2_margin":0.011684805154800415,"argmax_equal":false,"probs":[0.010005109012126923,0.39793699979782104,0.40590402483940125,0.1861538589000702],"probs_oracle":[0.009856987744569778,0.410186767578125,0.3985019624233246,0.18145422637462616]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":20,"questions":106,"argmax_equal_non_near_tie":"104/104","max_abs_dp":0.017072200775146484,"mean_of_run_mean_abs_dp":0.0008455247503376156},"g256":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.005918323993682861,"mean_of_run_mean_abs_dp":0.0005058204239670169},"g448":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.004572570323944092,"mean_of_run_mean_abs_dp":0.0003942434008330739}},"vs_python":{"all":{"runs":40,"hidden_bit_equal_runs":20,"hidden_max_abs_diff_where_unequal":3.1875,"max_abs_dp_vs_python_author_head":0.0014627575874328613},"by_arm":{"text":{"runs":20,"hidden_bit_equal_runs":20,"hidden_max_abs_diff_where_unequal":0.0,"max_abs_dp_vs_python_author_head":1.7881393432617188e-07},"g256":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.1875,"max_abs_dp_vs_python_author_head":0.0014627575874328613},"g448":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":1.8125,"max_abs_dp_vs_python_author_head":0.001261293888092041},"native":{"runs":0}},"python_readout":"readout_heldout_int8mix_pf64.json","python_readout_sha256":"2c0a1fd895a927c9f9ad0d0ca100d0d14d4a417c5baf1fbec15896395b945d2e","python_head_graph":null,"python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","name":"clef_flash_decode_int8mix_pf64","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":155,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the untied lm_head table stays on the host)"},"prefill_chunk":64,"metadata_sha256":"1eafaedf47981c83c82bf4f609615e2a33f28093b8fab3d89f86dec0e520fc36","main_mlirb":{"bytes":11826520241,"sha256":"2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_int8mix_pf64.h16c.aimodelc","bytes":25678382366,"tree_sha256":"e80f419a8a9af23390bd540270510413234361ef5f164659878f393f0b12681a","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"fc51b9aeb51a86c9fdb4061c6a6911def64e74a44008547d5e6e706d7423e4fe","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"5ac972e3a1bf9e3d91452395b2d3f874a3f2ce285694f7b50298522261226553","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f022d2d941944adfcfe6351d5ee94fc72b618eac18bd99ddb3935dc72ec6676f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"a4aef1c29efc3b7193da1c26b06495284c01311d7b4b12fa47e2b14a02ee847b","main-h16c.mlirb":"9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","main.hash":"3e20a00e80a801ac947f24ee223029c8688932ebb85e1c53369e5e42bed4ef82","metadata.json":"0d1604a74a754f6d673e475d48ce70591ab407fd83cf5a7a379055a56fd17c11","stats.json":"0c48677700237e6e276aff023c19d2ca080c23ab0b7b8631d3b959c83015c0e5"}}}},"runs":[{"id":"ho_i01","arm":"g256","source":"heldout_image","tokens":529,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0017434954643249512,"mean_abs_dp":0.0004008919573747195,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.2353515625,"vs_python_author_head_max_abs_dp":0.0001449882984161377},{"id":"ho_i01","arm":"g448","source":"heldout_image","tokens":661,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0003802180290222168,"mean_abs_dp":0.0001606022616705069,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.71484375,"vs_python_author_head_max_abs_dp":0.0001475512981414795},{"id":"ho_i02","arm":"g256","source":"heldout_image","tokens":570,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0004925727844238281,"mean_abs_dp":0.0001569937282641019,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3125,"vs_python_author_head_max_abs_dp":0.00010317564010620117},{"id":"ho_i02","arm":"g448","source":"heldout_image","tokens":702,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.001005709171295166,"mean_abs_dp":0.0002128587636564459,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":1.2265625,"vs_python_author_head_max_abs_dp":5.650520324707031e-05},{"id":"ho_i03","arm":"g256","source":"heldout_image","tokens":541,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0003185980021953583,"mean_abs_dp":5.828624125570059e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.234375,"vs_python_author_head_max_abs_dp":8.034706115722656e-05},{"id":"ho_i03","arm":"g448","source":"heldout_image","tokens":673,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00020659156143665314,"mean_abs_dp":4.144449485465884e-05,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.4140625,"vs_python_author_head_max_abs_dp":2.5818124413490295e-05},{"id":"ho_i04","arm":"g256","source":"heldout_image","tokens":437,"calls":7,"bucket":"t512","image_rows_from":"tower","max_abs_dp":0.005918323993682861,"mean_abs_dp":0.0012634898070245982,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.20703125,"vs_python_author_head_max_abs_dp":0.0006873607635498047},{"id":"ho_i04","arm":"g448","source":"heldout_image","tokens":569,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0033544600009918213,"mean_abs_dp":0.000755650014616549,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.296875,"vs_python_author_head_max_abs_dp":0.00016957521438598633},{"id":"ho_i05","arm":"g256","source":"heldout_image","tokens":701,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.004709124565124512,"mean_abs_dp":0.0009788602501863526,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.3984375,"vs_python_author_head_max_abs_dp":0.0005478858947753906},{"id":"ho_i05","arm":"g448","source":"heldout_image","tokens":833,"calls":14,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0012181401252746582,"mean_abs_dp":0.00030257927608631904,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":1.8125,"vs_python_author_head_max_abs_dp":0.00031638145446777344},{"id":"ho_i06","arm":"g256","source":"heldout_image","tokens":628,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0022852420806884766,"mean_abs_dp":0.0005009017247510584,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.33203125,"vs_python_author_head_max_abs_dp":0.0006378889083862305},{"id":"ho_i06","arm":"g448","source":"heldout_image","tokens":760,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.004572570323944092,"mean_abs_dp":0.0007792959038756395,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.25,"vs_python_author_head_max_abs_dp":0.001261293888092041},{"id":"ho_i07","arm":"g256","source":"heldout_image","tokens":535,"calls":9,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0017752796411514282,"mean_abs_dp":0.00031396123365714,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":3.1875,"vs_python_author_head_max_abs_dp":0.000357210636138916},{"id":"ho_i07","arm":"g448","source":"heldout_image","tokens":667,"calls":11,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.000686347484588623,"mean_abs_dp":0.0002533495247077483,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.375,"vs_python_author_head_max_abs_dp":0.00021705776453018188},{"id":"ho_i08","arm":"g256","source":"heldout_image","tokens":581,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.00157947838306427,"mean_abs_dp":0.0002848793588140432,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.28125,"vs_python_author_head_max_abs_dp":0.00028073787689208984},{"id":"ho_i08","arm":"g448","source":"heldout_image","tokens":713,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.002276599407196045,"mean_abs_dp":0.0006303908851216821,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.671875,"vs_python_author_head_max_abs_dp":0.00038039684295654297},{"id":"ho_j01","arm":"text","source":"heldout_json","tokens":769,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0032848715782165527,"mean_abs_dp":0.0007849081729849179,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_j02","arm":"text","source":"heldout_json","tokens":335,"calls":6,"bucket":"t512","image_rows_from":"none","max_abs_dp":0.004511445760726929,"mean_abs_dp":0.0015534954145550728,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":8.940696716308594e-08},{"id":"ho_j03","arm":"text","source":"heldout_json","tokens":775,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0013483166694641113,"mean_abs_dp":0.00046649506609690815,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_j04","arm":"text","source":"heldout_json","tokens":610,"calls":10,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0018457770347595215,"mean_abs_dp":0.0005143387243151665,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_j05","arm":"text","source":"heldout_json","tokens":775,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.010159105062484741,"mean_abs_dp":0.001323149977799724,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"ho_j06","arm":"text","source":"heldout_json","tokens":733,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.006981611251831055,"mean_abs_dp":0.001075278450217512,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"ho_j07","arm":"text","source":"heldout_json","tokens":826,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0009705424308776855,"mean_abs_dp":0.00017473678518500592,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"ho_j08","arm":"text","source":"heldout_json","tokens":798,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0072307586669921875,"mean_abs_dp":0.0014844793412420484,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_l01","arm":"text","source":"heldout_text_long","tokens":2850,"calls":45,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.004114627838134766,"mean_abs_dp":0.0006461821869015693,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_l02","arm":"text","source":"heldout_text_long","tokens":2731,"calls":43,"bucket":"t4096","image_rows_from":"none","max_abs_dp":0.0008147954940795898,"mean_abs_dp":0.00021711182004461685,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_p01","arm":"g256","source":"heldout_photo","tokens":634,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.001696944236755371,"mean_abs_dp":0.00041050941993792854,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.31640625,"vs_python_author_head_max_abs_dp":8.07642936706543e-05},{"id":"ho_p01","arm":"g448","source":"heldout_photo","tokens":766,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0025334954261779785,"mean_abs_dp":0.0005472399294376374,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.4921875,"vs_python_author_head_max_abs_dp":0.00012302398681640625},{"id":"ho_p02","arm":"g256","source":"heldout_photo","tokens":620,"calls":10,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0026553869247436523,"mean_abs_dp":0.0006894305184045259,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.231201171875,"vs_python_author_head_max_abs_dp":0.0014627575874328613},{"id":"ho_p02","arm":"g448","source":"heldout_photo","tokens":752,"calls":12,"bucket":"t1024","image_rows_from":"tower","max_abs_dp":0.0011062026023864746,"mean_abs_dp":0.0002590229543035521,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":false,"hidden_max_abs_diff_python":0.5,"vs_python_author_head_max_abs_dp":0.00018256902694702148},{"id":"ho_t01","arm":"text","source":"heldout_text","tokens":708,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0039014816284179688,"mean_abs_dp":0.0006967944586101701,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"ho_t02","arm":"text","source":"heldout_text","tokens":746,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.00368538498878479,"mean_abs_dp":0.0007297908014152199,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.7881393432617188e-07},{"id":"ho_t03","arm":"text","source":"heldout_text","tokens":842,"calls":14,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.001867055892944336,"mean_abs_dp":0.00040950073744170366,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"ho_t04","arm":"text","source":"heldout_text","tokens":658,"calls":11,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.002679169178009033,"mean_abs_dp":0.00038316170684993267,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_t05","arm":"text","source":"heldout_text","tokens":727,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0004647970199584961,"mean_abs_dp":0.00019458049376096045,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_t06","arm":"text","source":"heldout_text","tokens":721,"calls":12,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.0008625388145446777,"mean_abs_dp":0.0003672520465710584,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_t07","arm":"text","source":"heldout_text","tokens":1086,"calls":17,"bucket":"t2048","image_rows_from":"none","max_abs_dp":0.012249767780303955,"mean_abs_dp":0.0017647178169211436,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.1920928955078125e-07},{"id":"ho_t08","arm":"text","source":"heldout_text","tokens":864,"calls":14,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.017072200775146484,"mean_abs_dp":0.0020230136113241317,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":5.960464477539063e-08},{"id":"ho_t09","arm":"text","source":"heldout_text","tokens":867,"calls":14,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.007371366024017334,"mean_abs_dp":0.0013902967459418708,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":8.940696716308594e-08},{"id":"ho_t10","arm":"text","source":"heldout_text","tokens":800,"calls":13,"bucket":"t1024","image_rows_from":"none","max_abs_dp":0.003891885280609131,"mean_abs_dp":0.0007112106485735802,"finite":true,"argmax_equal_non_near_tie":true,"softmax_recomputed_bit_equal":true,"response_recomputed_equal":true,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_sha256_equal_python":true,"vs_python_author_head_max_abs_dp":1.4901161193847656e-07}]}},"oracle_rows_pass":{"fixture_json":["<lane>/swift/gate/int8mix_aot/fixture_rows.json","<lane>/swift/gate/int8mix_aot/heldout_rows.json"],"what":"g256 / g448 runs fed the oracle's fp32 rows (the readout_gate.py inputs): the Swift decoder + head vs the Python decoder + head on identical inputs","reset_checks":[{"run":"img_01/g256","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":0.950015},{"run":"ho_i01/g256","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":1.217612709}],"fixture":{"runs":28,"hidden_bit_equal":28,"logits_bit_equal_head_graph":"n/a (no Python head-graph run on this decoder's rows)","max_abs_dp_vs_python_author_head":1.1920928955078125e-07,"rows":[{"id":"img_01","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_01","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_02","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_02","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_03","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_03","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_04","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_04","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":2.2351741790771484e-08},{"id":"img_05","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":3.166496753692627e-08},{"id":"img_05","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_06","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":8.940696716308594e-08},{"id":"img_06","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_07","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_07","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_08","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"img_08","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_09","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"img_09","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.4901161193847656e-08},{"id":"img_10","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":2.60770320892334e-08},{"id":"img_10","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":2.2351741790771484e-08},{"id":"photo_01","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_01","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":2.9802322387695312e-08},{"id":"photo_02","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_02","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":2.2351741790771484e-08},{"id":"photo_03","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_03","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"photo_04","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":9.313225746154785e-09},{"id":"photo_04","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":6.51925802230835e-09}]},"heldout":{"runs":20,"hidden_bit_equal":20,"logits_bit_equal_head_graph":"n/a (no Python head-graph run on this decoder's rows)","max_abs_dp_vs_python_author_head":2.384185791015625e-07,"rows":[{"id":"ho_i01","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":7.450580596923828e-09},{"id":"ho_i01","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i02","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.6763806343078613e-08},{"id":"ho_i02","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i03","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i03","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i04","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i04","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i05","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i05","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"ho_i06","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":7.450580596923828e-08},{"id":"ho_i06","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i07","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08},{"id":"ho_i07","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i08","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_i08","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_p01","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_p01","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":1.1920928955078125e-07},{"id":"ho_p02","arm":"g256","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":2.384185791015625e-07},{"id":"ho_p02","arm":"g448","hidden_bit_equal_python":true,"max_abs_dp_vs_python_author_head":5.960464477539063e-08}]}},"combined":{"runs":253,"questions_non_near_tie":583,"argmax_equal_non_near_tie":583,"near_tie_questions":6,"argmax_equal_near_tie":5,"max_abs_dp":0.017072200775146484,"result":"PASS"}},"int8mix_jit":{"schema":"clef-flash-swift-gate/1","asset":"int8mix_jit","asset_note":"the .aimodel specialized by Swift (GPU preferred + expectFrequentReshapes)","gate":"Swift host (apps/ClefFlash, clef-flash CLI) vs the author's fp32 oracle, and vs the Python reference of the same assets","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"source":"readout_gate.BAR (unchanged)"},"package":{"path":"<zoo>/apps/ClefFlash","files_sha256":{"Package.swift":"17bc993477be7cadb01e46a4a05d21e4b899ee29d360a8005dbf28662b7bda98","README.md":"80b4fde7bb164ff636896e9ad4487d646dc4c0928310dd2d682759c9709d9ffc","Sources/ClefFlash/ClefDecider.swift":"e52a1d3b3cd4d11e64146824b069bdb32d3c017d2db6cbff54e3e47d9135b2b9","Sources/ClefFlash/Decoder.swift":"2ee502b96977cd3ea14a2668d791d073d7978870ad0f3ba43bc86527554835cd","Sources/ClefFlash/Head.swift":"671fc5adbe34427471f41e97dc9b87c6bea489ed93d42653208e23b697394c89","Sources/ClefFlash/ImagePreprocess.swift":"6f18f0592fb6accf63f6aee11dd8106812db0297ac44aba38ab6b9a253364ebc","Sources/ClefFlash/JSONValue.swift":"bb184d942a66934e8cea895c0ddb05e988ba00cec9b509eafe5ce5d7571039bf","Sources/ClefFlash/PromptBuilder.swift":"7d5fa87d532d199022339698f8e14bce8a721e4aefda0dfdc5ce5272ff047110","Sources/ClefFlash/Renderer.swift":"963e0ff25b217b19f7be86f6bf6da5c2c2781649d52a13c069f7be1841a8ce49","Sources/ClefFlash/Support.swift":"a96745ace5cdde3d7e55001bf134e12c9ff067e7f18e68aeb7011fb44729e85b","Sources/ClefFlash/SystemOne.swift":"12e38a5829f073d4515bbeb64dff4a1992b190a0632a5db1cfb52ec28326d771","Sources/ClefFlash/VisionTower.swift":"4f1b3e8d1d97bb18366460f9548862e15f33d6b8ed999456ad8eb5cfd5de456d","Sources/clef-flash/main.swift":"50384ac0836684fa4795e6d2c2dd088038255c98185b6ed6ed66ec9e4a4c433d","_time_mac.sh":"b4f36c9b5b9e9ccd30f26c1b7db94f197d84f1b48b4252cc0af61f40c4326726"},"resolved":{"eventsource":"1.5.1","swift-asn1":"1.7.3","swift-collections":"1.7.1","swift-crypto":"4.5.2","swift-huggingface":"0.12.0","swift-jinja":"2.5.1","swift-transformers":"1.3.4","yyjson":"0.12.0"},"binary":{"path":"<lane>/swift/.build/release/clef-flash","sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","bytes":8600088}},"generated_at":"2026-10-03T06:36:06+00:00","passes":{"fixture":{"set":"fixture","fixture_json":"<lane>/swift/gate/int8mix_jit/fixture.json","label":"r6 int8mix_jit fixture","assets":{"asset":"jit","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","decoder":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64/clef_flash_decode_int8mix_pf64.aimodel","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.aimodel","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32/clef_flash_g256_vision_fp16w32.aimodel","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32/clef_flash_g448_vision_fp16w32.aimodel"},"decoder_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes true","head_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false","tower_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false"},"binary_sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","load_first":{"wall_s":42.296937459,"tokenizer_s":0.644797916,"decoder_s":{"model":37.395256917,"main":1.802646083},"head_s":0.304128958,"tower_s":{"g256":1.10806175,"g448":1.03289125}},"reset_check":{"run":"own_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":1.036022792},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":89275039744,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":106307362816,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":106307362816,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"photo_01","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{"photo_01/native":"40 x 30 = 1,200 image rows > the graph's 1,024-row buffer (as in readout_gate.py)"},"summary":{"runs":213,"expected_runs":213,"questions":401,"questions_non_near_tie":397,"argmax_equal_non_near_tie":397,"near_tie_questions":4,"argmax_equal_near_tie":4,"max_abs_dp":0.014843374490737915,"mean_of_run_mean_abs_dp":0.0008873040920171905,"max_abs_dlogit_vs_oracle":0.28801047801971436,"worst_run":{"id":"own_t03","arm":"text","max_abs_dp":0.014843374490737915},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":213,"spans_equal_oracle":213,"softmax_recomputed_bit_equal_runs":213,"response_recomputed_equal_runs":213,"near_ties":[{"id":"img_01","arm":"g256","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.4941219687461853,0.4848661422729492,0.02101188153028488],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_01","arm":"native","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"probs":[0.49450868368148804,0.4844876527786255,0.021003684028983116],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_06","arm":"g448","question_id":"in_red_zone","oracle_top2_margin":0.017518073320388794,"argmax_equal":true,"probs":[0.5130700469017029,0.4869299829006195],"probs_oracle":[0.5087590217590332,0.4912409484386444]},{"id":"photo_01","arm":"g256","question_id":"kinds","oracle_top2_margin":0.00908571481704712,"argmax_equal":true,"probs":[0.023645097389817238,0.11538858711719513,0.3806717097759247,0.3905435800552368,0.08975104242563248],"probs_oracle":[0.023699797689914703,0.11542528122663498,0.3812732696533203,0.39035898447036743,0.08924264460802078]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":172,"questions":255,"argmax_equal_non_near_tie":"255/255","max_abs_dp":0.014843374490737915,"mean_of_run_mean_abs_dp":0.0009698852109839036},"g256":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"48/48","max_abs_dp":0.013218581676483154,"mean_of_run_mean_abs_dp":0.0007174685753946688},"g448":{"runs":14,"questions":50,"argmax_equal_non_near_tie":"49/49","max_abs_dp":0.004977524280548096,"mean_of_run_mean_abs_dp":0.0004926791996181908},"native":{"runs":13,"questions":46,"argmax_equal_non_near_tie":"45/45","max_abs_dp":0.0039743781089782715,"mean_of_run_mean_abs_dp":0.000402572804634624}},"vs_python":{"all":{"runs":213,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.625,"max_abs_dp_vs_python_author_head":0.0014283061027526855},"by_arm":{"text":{"runs":172,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":0.28125,"max_abs_dp_vs_python_author_head":0.0014283061027526855},"g256":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":0.828125,"max_abs_dp_vs_python_author_head":0.0011033415794372559},"g448":{"runs":14,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.625,"max_abs_dp_vs_python_author_head":0.0004334747791290283},"native":{"runs":13,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":0.251953125,"max_abs_dp_vs_python_author_head":0.00019240379333496094}},"python_readout":"readout_int8mix_pf64.json","python_readout_sha256":"e7eb22c1affea851f67158594029b7ff7a4369bcc45a55421dcb55c8ed5e6b2f","python_head_graph":null,"python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","name":"clef_flash_decode_int8mix_pf64","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":155,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the untied lm_head table stays on the host)"},"prefill_chunk":64,"metadata_sha256":"1eafaedf47981c83c82bf4f609615e2a33f28093b8fab3d89f86dec0e520fc36","main_mlirb":{"bytes":11826520241,"sha256":"2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_int8mix_pf64.h16c.aimodelc","bytes":25678382366,"tree_sha256":"e80f419a8a9af23390bd540270510413234361ef5f164659878f393f0b12681a","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"fc51b9aeb51a86c9fdb4061c6a6911def64e74a44008547d5e6e706d7423e4fe","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"5ac972e3a1bf9e3d91452395b2d3f874a3f2ce285694f7b50298522261226553","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f022d2d941944adfcfe6351d5ee94fc72b618eac18bd99ddb3935dc72ec6676f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"a4aef1c29efc3b7193da1c26b06495284c01311d7b4b12fa47e2b14a02ee847b","main-h16c.mlirb":"9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","main.hash":"3e20a00e80a801ac947f24ee223029c8688932ebb85e1c53369e5e42bed4ef82","metadata.json":"0d1604a74a754f6d673e475d48ce70591ab407fd83cf5a7a379055a56fd17c11","stats.json":"0c48677700237e6e276aff023c19d2ca080c23ab0b7b8631d3b959c83015c0e5"}}}}},"heldout":{"set":"heldout","fixture_json":"<lane>/swift/gate/int8mix_jit/heldout.json","label":"r6 int8mix_jit heldout","assets":{"asset":"jit","decoder_bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","decoder":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64/clef_flash_decode_int8mix_pf64.aimodel","head":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_asset":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.aimodel","table":"<lane>/exports/host/lm_head_fp16.bin","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32/clef_flash_g256_vision_fp16w32.aimodel","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32/clef_flash_g448_vision_fp16w32.aimodel"},"decoder_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes true","head_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false","tower_options":"preferred gpu, allowed [\"cpu\", \"gpu\", \"neuralEngine\"], expectFrequentReshapes false"},"binary_sha256":"96882c4ff0f981862ecc4aa59a55adb6982b69203d9e1f1c518aad12fb1fd65e","load_first":{"wall_s":2.966405833,"tokenizer_s":0.672493125,"decoder_s":{"model":0.000844834,"main":1.902557459},"head_s":0.064429583,"tower_s":{"g256":0.1783085,"g448":0.13855762500000002}},"reset_check":{"run":"ho_t01/text","hidden_bit_equal":true,"logits_bit_equal":true,"wall_s":1.774852208},"coreai_cache":{"coreai_cache_before_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":106307362816,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_after_load":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":106307362816,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]},"coreai_cache_end":{"path":"~/Library/Caches/coreai-cache/<os build>/clef-flash","bytes":106307362816,"entries":["04d8629bedeb1af76a2e63ed677a931c187289a6478fc66bf33a7bfa523a1eaa","0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8","0d4410d326afe6010f089f9e71c9187a5ac82e0aafae2aad4aed9d3fc6abec68","0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9","51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe","9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","b2adb0396f6f37f3178c0d7b413d10cddc31271c0dc7a56b2416125e9664f31d","d814131b0c9015da52d207b35f7dc678d1567a4341b0989ebb35290e26bd8a9d"]}},"skipped_by_swift":[{"id":"ho_i01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i02","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i03","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i04","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i05","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i06","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i07","arm":"native","why":"no image rows in --embeds"},{"id":"ho_i08","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p01","arm":"native","why":"no image rows in --embeds"},{"id":"ho_p02","arm":"native","why":"no image rows in --embeds"}],"skipped_by_gate":{},"summary":{"runs":40,"expected_runs":40,"questions":188,"questions_non_near_tie":186,"argmax_equal_non_near_tie":186,"near_tie_questions":2,"argmax_equal_near_tie":1,"max_abs_dp":0.017312049865722656,"mean_of_run_mean_abs_dp":0.000647736907568411,"max_abs_dlogit_vs_oracle":0.11842536926269531,"worst_run":{"id":"ho_t08","arm":"text","max_abs_dp":0.017312049865722656},"finite_all":true,"reset_bit_equal":true,"ids_equal_oracle":40,"spans_equal_oracle":40,"softmax_recomputed_bit_equal_runs":40,"response_recomputed_equal_runs":40,"near_ties":[{"id":"ho_j01","arm":"text","question_id":"late_start","oracle_top2_margin":0.0010575354099273682,"argmax_equal":true,"probs":[0.5017731189727783,0.4982268512248993],"probs_oracle":[0.5005287528038025,0.4994712173938751]},{"id":"ho_t07","arm":"text","question_id":"questions_asked","oracle_top2_margin":0.011684805154800415,"argmax_equal":false,"probs":[0.010006597265601158,0.39823290705680847,0.40569961071014404,0.18606095016002655],"probs_oracle":[0.009856987744569778,0.410186767578125,0.3985019624233246,0.18145422637462616]}]},"checks":{"all_runs":true,"ids_spans_rope_usage":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal":true},"result":"PASS","g1_failures":[],"per_arm":{"text":{"runs":20,"questions":106,"argmax_equal_non_near_tie":"104/104","max_abs_dp":0.017312049865722656,"mean_of_run_mean_abs_dp":0.000848636746548336},"g256":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.005705446004867554,"mean_of_run_mean_abs_dp":0.0004914691183141129},"g448":{"runs":10,"questions":41,"argmax_equal_non_near_tie":"41/41","max_abs_dp":0.004756718873977661,"mean_of_run_mean_abs_dp":0.000402205018862859}},"vs_python":{"all":{"runs":40,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.140625,"max_abs_dp_vs_python_author_head":0.0014454424381256104},"by_arm":{"text":{"runs":20,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":0.296875,"max_abs_dp_vs_python_author_head":0.0005339086055755615},"g256":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":3.140625,"max_abs_dp_vs_python_author_head":0.0013720393180847168},"g448":{"runs":10,"hidden_bit_equal_runs":0,"hidden_max_abs_diff_where_unequal":1.75,"max_abs_dp_vs_python_author_head":0.0014454424381256104},"native":{"runs":0}},"python_readout":"readout_heldout_int8mix_pf64.json","python_readout_sha256":"2c0a1fd895a927c9f9ad0d0ca100d0d14d4a417c5baf1fbec15896395b945d2e","python_head_graph":null,"python_bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_int8mix_pf64","name":"clef_flash_decode_int8mix_pf64","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":155,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the untied lm_head table stays on the host)"},"prefill_chunk":64,"metadata_sha256":"1eafaedf47981c83c82bf4f609615e2a33f28093b8fab3d89f86dec0e520fc36","main_mlirb":{"bytes":11826520241,"sha256":"2c19a0e765a659e0f726b4b7099a62490fed6ab2e4db1246eb63fd70996a67b9"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_int8mix_pf64.h16c.aimodelc","bytes":25678382366,"tree_sha256":"e80f419a8a9af23390bd540270510413234361ef5f164659878f393f0b12681a","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"fc51b9aeb51a86c9fdb4061c6a6911def64e74a44008547d5e6e706d7423e4fe","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"5ac972e3a1bf9e3d91452395b2d3f874a3f2ce285694f7b50298522261226553","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f022d2d941944adfcfe6351d5ee94fc72b618eac18bd99ddb3935dc72ec6676f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"a4aef1c29efc3b7193da1c26b06495284c01311d7b4b12fa47e2b14a02ee847b","main-h16c.mlirb":"9bce40915b0251124de6bbed06bfd64755fde5ba20a11c29bcdfdf5784468ab2","main.hash":"3e20a00e80a801ac947f24ee223029c8688932ebb85e1c53369e5e42bed4ef82","metadata.json":"0d1604a74a754f6d673e475d48ce70591ab407fd83cf5a7a379055a56fd17c11","stats.json":"0c48677700237e6e276aff023c19d2ca080c23ab0b7b8631d3b959c83015c0e5"}}}}}},"same_as":{"other_passes":["<lane>/swift/gate/int8mix_aot/fixture.json","<lane>/swift/gate/int8mix_aot/heldout.json"],"runs_compared":253,"hidden_bit_equal":0,"logits_bit_equal":0,"image_rows_bit_equal":"61/61","responses_equal":80},"combined":{"runs":253,"questions_non_near_tie":583,"argmax_equal_non_near_tie":583,"near_tie_questions":6,"argmax_equal_near_tie":5,"max_abs_dp":0.017312049865722656,"result":"PASS"}},"jit_vs_aot":{"fp16":{"what":"fp16: the Swift JIT passes vs the Swift AOT passes of the same bundle, every option of every question, computed in round 7 from the pass files below (logits and float32 probabilities as the CLI wrote them); no rerun","pass_files":[{"file":"swift/gate/fp16_aot/fixture.json","bytes":1363023,"sha256":"7aaa9ea6f2bd270228ee1df1c25d7bd9508ec577fd833d04705cc7fdd4f214ee"},{"file":"swift/gate/fp16_jit/fixture.json","bytes":1363798,"sha256":"5d5524cb9493aa32bb59befdf6a515adf798e623bd66542d50aaef534f43dd26"},{"file":"swift/gate/fp16_aot/heldout.json","bytes":546629,"sha256":"8316d67c4c38ce4df9b168f58e58efb2d962a3a6f929c5f03dee43ea37a494cf"},{"file":"swift/gate/fp16_jit/heldout.json","bytes":547584,"sha256":"6cb3dadeaa2fec2e9c52121d241c41185f64fa4fc7df0c4ab113ca9ee0988b6f"}],"runs":253,"questions":589,"argmax_equal":589,"max_abs_dp":0.0,"max_abs_dlogit":0.0,"logits_bit_equal_runs":253,"responses_equal":253},"int8mix":{"what":"int8mix: the Swift JIT passes vs the Swift AOT passes of the same bundle, every option of every question, computed in round 7 from the pass files below (logits and float32 probabilities as the CLI wrote them); no rerun","pass_files":[{"file":"swift/gate/int8mix_aot/fixture.json","bytes":1364328,"sha256":"5cc50715319cafecec5cb90869db99120db80c1e017e09c804d64a47449fc3a9"},{"file":"swift/gate/int8mix_jit/fixture.json","bytes":1365163,"sha256":"dcaedfe89b18b60dccd301fb9374cd29157ff8ac9c4e9f9c73b8b60d2cc44790"},{"file":"swift/gate/int8mix_aot/heldout.json","bytes":547553,"sha256":"44a7f0f0702f6325f9884ec4795335092b3eb48c70078458f28f074825304c37"},{"file":"swift/gate/int8mix_jit/heldout.json","bytes":548244,"sha256":"f4a254a25495f28989e19a89dac60279fc4729aa965c38d314e9250daca1107b"}],"runs":253,"questions":589,"argmax_equal":589,"max_abs_dp":0.001428365707397461,"max_abs_dlogit":0.020124435424804688,"logits_bit_equal_runs":0,"responses_equal":80}},"render_check":{"schema":"clef-flash-swift-render/1","generated_at":"2026-10-03T05:58:31+00:00","what":"PythonJSON (apps/ClefFlash Renderer.swift) through `clef-flash render-test` vs CPython 3.11.13 json.dumps(ensure_ascii=False, separators=(',', ':'), sort_keys=True), repr(float), round(x, 4)","render":{"n":2204,"equal":2204,"count_match":true,"first_differences":[]},"canonical":{"n":2204,"equal":2204,"count_match":true,"first_differences":[]},"raw_literals":{"n":17,"equal":17,"count_match":true,"first_differences":[]},"float_repr":{"n":199994,"equal":199994,"count_match":true,"first_differences":[]},"round4":{"n":199994,"equal":199994,"count_match":true,"first_differences":[]},"values":2204,"request_values":2204,"doubles":199994,"pass":true},"pixels":{"schema":"clef-flash-swift-pixels/1","generated_at":"2026-10-03T05:58:41+00:00","sets":{"fixture":{"summary":{"tiles":28,"decoded_rgb_equal_pillow":28,"tiles_bit_equal_pillow":28,"max_abs_level":0,"patches_equal_host_preprocess":28,"oracle_tiles":28,"patches_equal_oracle_pixel_values":28},"tiles":[{"image":"arrows_640x480.png","grid":"g256","size_in":[640,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"arrows_640x480.png","grid":"g448","size_in":[640,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"bars_640x480.png","grid":"g256","size_in":[640,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"bars_640x480.png","grid":"g448","size_in":[640,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"circles_640x480.png","grid":"g256","size_in":[640,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"circles_640x480.png","grid":"g448","size_in":[640,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"clock_3oclock_256x240.png","grid":"g256","size_in":[256,240],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"clock_3oclock_256x240.png","grid":"g448","size_in":[256,240],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"floorplan_1024x768.png","grid":"g256","size_in":[1024,768],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"floorplan_1024x768.png","grid":"g448","size_in":[1024,768],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"gauge_300x200.png","grid":"g256","size_in":[300,200],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"gauge_300x200.png","grid":"g448","size_in":[300,200],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_cat.png","grid":"g256","size_in":[960,640],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_cat.png","grid":"g448","size_in":[960,640],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_coffee.png","grid":"g256","size_in":[960,635],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_coffee.png","grid":"g448","size_in":[960,635],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_fruit.png","grid":"g256","size_in":[960,1280],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_fruit.png","grid":"g448","size_in":[960,1280],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_lake.png","grid":"g256","size_in":[960,640],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"photo_lake.png","grid":"g448","size_in":[960,640],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"receipt_1024x768.png","grid":"g256","size_in":[1024,768],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"receipt_1024x768.png","grid":"g448","size_in":[1024,768],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"sevenseg_47_300x200.png","grid":"g256","size_in":[300,200],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"sevenseg_47_300x200.png","grid":"g448","size_in":[300,200],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"shapes_224.png","grid":"g256","size_in":[224,224],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"shapes_224.png","grid":"g448","size_in":[224,224],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"signal_green_256x240.png","grid":"g256","size_in":[256,240],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"signal_green_256x240.png","grid":"g448","size_in":[256,240],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true}]},"heldout":{"summary":{"tiles":20,"decoded_rgb_equal_pillow":20,"tiles_bit_equal_pillow":20,"max_abs_level":0,"patches_equal_host_preprocess":20,"oracle_tiles":20,"patches_equal_oracle_pixel_values":20},"tiles":[{"image":"ho_calendar_760x700.png","grid":"g256","size_in":[760,700],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_calendar_760x700.png","grid":"g448","size_in":[760,700],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_digits_960x240.png","grid":"g256","size_in":[960,240],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_digits_960x240.png","grid":"g448","size_in":[960,240],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_floorplan_1200x900.png","grid":"g256","size_in":[1200,900],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_floorplan_1200x900.png","grid":"g448","size_in":[1200,900],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_photo_lighthouse.png","grid":"g256","size_in":[960,960],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_photo_lighthouse.png","grid":"g448","size_in":[960,960],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_photo_sunflowers.png","grid":"g256","size_in":[960,1280],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_photo_sunflowers.png","grid":"g448","size_in":[960,1280],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_pie_720x480.png","grid":"g256","size_in":[720,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_pie_720x480.png","grid":"g448","size_in":[720,480],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_receipt_480x840.png","grid":"g256","size_in":[480,840],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_receipt_480x840.png","grid":"g448","size_in":[480,840],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_sign_200x150.png","grid":"g256","size_in":[200,150],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_sign_200x150.png","grid":"g448","size_in":[200,150],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_table_1000x600.png","grid":"g256","size_in":[1000,600],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_table_1000x600.png","grid":"g448","size_in":[1000,600],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_thermometer_160x400.png","grid":"g256","size_in":[160,400],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true},{"image":"ho_thermometer_160x400.png","grid":"g448","size_in":[160,400],"decode_path":"direct RGBX","decoded_rgb_equal_pillow":true,"tile_max_abs_level_vs_pillow":0,"tile_values_differing":0,"patches_sha256_equal_host":true,"patches_sha256_equal_oracle_pixel_values":true}]}},"pass":true},"towers":{"schema":"clef-flash-swift-towers/1","generated_at":"2026-10-03T06:07:52+00:00","towers":{"g256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","g448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"what":"Swift VisionTower output (dumped by the gate pass) vs the Python runtime (coreai.runtime, SpecializationOptions.default) on host.preprocess (Pillow) patches, same AOT asset","runs":48,"bit_equal":48,"patches_equal":48,"max_abs_diff":0.0,"rows":[{"id":"img_01","arm":"g256","image":"shapes_224.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_01","arm":"g448","image":"shapes_224.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_02","arm":"g256","image":"bars_640x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_02","arm":"g448","image":"bars_640x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_03","arm":"g256","image":"sevenseg_47_300x200.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_03","arm":"g448","image":"sevenseg_47_300x200.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_04","arm":"g256","image":"receipt_1024x768.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_04","arm":"g448","image":"receipt_1024x768.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_05","arm":"g256","image":"signal_green_256x240.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_05","arm":"g448","image":"signal_green_256x240.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_06","arm":"g256","image":"gauge_300x200.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_06","arm":"g448","image":"gauge_300x200.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_07","arm":"g256","image":"arrows_640x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_07","arm":"g448","image":"arrows_640x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_08","arm":"g256","image":"floorplan_1024x768.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_08","arm":"g448","image":"floorplan_1024x768.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_09","arm":"g256","image":"circles_640x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_09","arm":"g448","image":"circles_640x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_10","arm":"g256","image":"clock_3oclock_256x240.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"img_10","arm":"g448","image":"clock_3oclock_256x240.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_01","arm":"g256","image":"photo_fruit.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_01","arm":"g448","image":"photo_fruit.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_02","arm":"g256","image":"photo_cat.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_02","arm":"g448","image":"photo_cat.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_03","arm":"g256","image":"photo_lake.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_03","arm":"g448","image":"photo_lake.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_04","arm":"g256","image":"photo_coffee.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"photo_04","arm":"g448","image":"photo_coffee.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i01","arm":"g256","image":"ho_pie_720x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i01","arm":"g448","image":"ho_pie_720x480.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i02","arm":"g256","image":"ho_table_1000x600.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i02","arm":"g448","image":"ho_table_1000x600.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i03","arm":"g256","image":"ho_sign_200x150.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i03","arm":"g448","image":"ho_sign_200x150.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i04","arm":"g256","image":"ho_thermometer_160x400.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i04","arm":"g448","image":"ho_thermometer_160x400.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i05","arm":"g256","image":"ho_calendar_760x700.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i05","arm":"g448","image":"ho_calendar_760x700.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i06","arm":"g256","image":"ho_digits_960x240.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i06","arm":"g448","image":"ho_digits_960x240.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i07","arm":"g256","image":"ho_receipt_480x840.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i07","arm":"g448","image":"ho_receipt_480x840.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i08","arm":"g256","image":"ho_floorplan_1200x900.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_i08","arm":"g448","image":"ho_floorplan_1200x900.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_p01","arm":"g256","image":"ho_photo_lighthouse.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_p01","arm":"g448","image":"ho_photo_lighthouse.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_p02","arm":"g256","image":"ho_photo_sunflowers.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true},{"id":"ho_p02","arm":"g448","image":"ho_photo_sunflowers.png","bit_equal":true,"max_abs_diff":0.0,"patches_sha256_equal":true}]},"negative_control":{"schema":"clef-flash-swift-negative/1","generated_at":"2026-10-03T06:34:31+00:00","run":"own_t01/text","base_trace":"<lane>/swift/ask/own_t01_fp16_aot.trace.json","changed_trace":"<lane>/swift/ask/neg_own_t01_one_word.trace.json","changed_request":"<lane>/swift/requests/neg_own_t01_one_word.json","tokens":[436,440],"base_vs_oracle":{"ids_equal":true,"spans_equal":true},"changed_vs_oracle":{"ids_equal":false,"spans_equal":false},"questions":[{"question_id":"department","max_abs_dp":0.0037911534309387207,"argmax":[3,3],"p_base":[0.002065346809104085,0.009691590443253517,0.009932344779372215,0.9783106446266174],"p_changed":[0.0019426976796239614,0.008437455631792545,0.007517991587519646,0.9821017980575562],"question_span":[[124,131],[124,135]]},{"question_id":"wants_refund","max_abs_dp":0.00010354537516832352,"argmax":[1,1],"p_base":[0.013661152683198452,0.9863388538360596],"p_changed":[0.013557607308030128,0.9864423274993896],"question_span":[[256,263],[260,267]]},{"question_id":"urgency","max_abs_dp":0.005634427070617676,"argmax":[1,1],"p_base":[0.30850479006767273,0.3807034492492676,0.2241351157426834,0.08665654808282852],"p_changed":[0.3141392171382904,0.37693798542022705,0.2186153531074524,0.09030743688344955],"question_span":[[337,343],[341,347]]}],"max_abs_dp":0.005634427070617676,"red":true}}