{"what":"The joint schema head as a graph (conversion/clef_flash/clef_head.py ClefHeadGraph, fp16 weights read through a cast, fp32 math). h0: the matrix form in eager fp32 against the author's JointSchemaHead. h1: the AOT head graph on the oracle's fp32 hidden. h2: end to end in Python from the decoder's fp16 hidden (readout transcripts) through the host arrays and the head graph. decide_py: conversion/clef_flash/decide.py from the raw request against the author's systemone() response. export: the shipped bucket asset and the dynamic-shape asset that aborts at its first call. key_block_probes: why attention over 4,032 or more keys is computed in key blocks. diag_long: one 4,050-token row (a diagnostic, not a gate set) through the decoder, the head and decide.py.","sources":[{"file":"results/head_h0.json","bytes":170092,"sha256":"1f8c472af008a921bf4a09d804448eb17bd7519d5cc4c4fd314608866838a27e"},{"file":"results/head_h1.json","bytes":468631,"sha256":"d946c24fbd458ff4662376a8001d7264de5e885ee2994eba83b21d586dd947ac"},{"file":"results/head_h1_heldout.json","bytes":178557,"sha256":"60bdea66ec898167ba3550bb971b01f9dc9d8e68fc5729937fb4d7965afb1cff"},{"file":"results/head_h2_fixture.json","bytes":525974,"sha256":"5445ae274aea8119d16264d0e8886295cd76ed0a6f072579d17835eb57c7f7b9"},{"file":"results/head_h2_heldout.json","bytes":189430,"sha256":"1a70f7ddfe7525c9b1cbba6f5a1d2b2456338fb88955b72cb1d8dba4dd04e736"},{"file":"results/head_h2_diag_long.json","bytes":17923,"sha256":"4c13e37f6d73fd7cd53b176a57236a094643551fdbfba4d7d362e07fbd4619e9"},{"file":"results/decide_e2e_fixture.json","bytes":64906,"sha256":"230fd1d03c56c55ffb3c6f4c8a63f7802c87d6cdef9b09b7d215695bdf1db6fb"},{"file":"results/decide_e2e_heldout.json","bytes":81232,"sha256":"ce47d3b4d8526a7659bef011d64ed4b61d6d97a264d14dedee4663767f325352"},{"file":"results/decide_e2e_diag_long.json","bytes":9371,"sha256":"df57b3b31809104f0d271edfb59f5ac88664c9d2de0c7244a7738a58dd733fe0"},{"file":"results/readout_diag_long_fp16_pf64.json","bytes":19066,"sha256":"bc651ee470dbec4a2e1c19485766decdd0529194549723d747877d158a690b7e"},{"file":"logs/r5_export_head_bucket.json","bytes":5303,"sha256":"0ef0f2b4012265f77ee9a064d800f8d1764ab956b7ef67b0e4f8c6c4937cb51f"},{"file":"logs/r5_export_head_dynamic.json","bytes":5703,"sha256":"01174f4009169a1241595f026ad781f98fe3caf92429abfc4c8eb48a9db6bff4"},{"file":"logs/r5_h1.log","bytes":715,"sha256":"369a2cbefe88d2c70b847dec399b0137ade8eb8816bd94c2533a97fdbe438548"},{"file":"logs/r5_dynamic_probe.log","bytes":1698,"sha256":"1c394c2250db92fbf31b7d704a0c17e7b75eadb9a15d46d5d2f268d8c3ac3578"},{"file":"logs/r5_dynamic_reuse_probe.log","bytes":530,"sha256":"ba571d07d792af7ad11439e7d2280b86c08d423d0b2ff94faa2dfb1d60cd7f56"},{"file":"logs/r5_t4096_probe.log","bytes":1157,"sha256":"5ba1c59996c54c1a0be293578aef8aa9f8295739fe67a2e664d9c6ce19490e8a"},{"file":"logs/r5_tmax_probe.log","bytes":1672,"sha256":"aa9cca01dd5bd12955cc230830961b0c521e941648e9a373b0aada36f98b987c"},{"file":"logs/r5_head_debug.log","bytes":1500,"sha256":"c6a0ae5d0d995956fa302daebae448117813d0d7a179081c54596765ae399f0d"},{"file":"logs/r5_attn_probe.log","bytes":1561,"sha256":"e52d87c7025b4527203a19adb978cb7c1e676f0a339f93a5f384c526cf2d80d6"},{"file":"logs/r5_attn_fix_probe.log","bytes":1085,"sha256":"e7ddd86cf70b1714e78df5a95fc18380db83267f726674a1ec7528b68d4aa090"},{"file":"logs/r5_op_probe.log","bytes":1407,"sha256":"ed1ac12399d5667336bc1311491aebb0d9e74adff0741dc474de6041d93f3aa6"},{"file":"results/head_op_census.json","bytes":3524,"sha256":"77a31d7cd0fc9198fdbbcf0e0ba85fe97f33821e5ae4c7380c5e52c880b607cc"},{"file":"results/head_export_scout.json","bytes":17393,"sha256":"6b12c81af89f41ea8bd0fc9ea59d0e34f3e1a88d0c2eaa9218a0e36265f1370b"},{"file":"results/head_header.json","bytes":16727,"sha256":"c08837ad4d6edd0da2be446397009e3fb196ee2b15f4729a3d83f96fe0f190cd"}],"trimmed":{"removed":["h0: nothing (per-run exact / bucket logit differences kept)","h1 / h2: runs[].questions (per-question probabilities; each run keeps max / mean |dp|, max |dlogit| and the argmax flags)","h1 / h2: runs[].logits, runs[].call_ms","h1 / h2: processes[].descriptors (= descriptors; descriptors_same_every_process)","diag_long.decoder_readout: runs[].call_ms (per-call ms; the timing block keeps the medians)","diag_long.decoder_readout: runs[].npz / hidden_npz / npz_key (lane file names)","diag_long.decoder_readout: runs[].hidden.lowest_positions (each run keeps min_pos_cos, its index and the counts below 0.99 / 0.9 / 0.5)","diag_long.decoder_readout: runs[].questions[].probs_oracle (the oracle's probabilities are in fixtures-clef-flash.json)","diag_long.decoder_readout: oracle.npz_sha256 (sha256 of the lane's per-run oracle npz files)","diag_long.decoder_readout: processes[].npz / npz_sha256 / descriptor (the descriptor is the top-level one: descriptor_same_in_every_process)","head_header.tensors (per-tensor names and shapes of joint_head.safetensors; by_module and totals kept)"],"serialization":"compact json (no indentation)","full_copies":"the source files above, in the lane directory $ZOO_WORK_ROOT/_clefflash/ (not in the repository)"},"h0":{"schema":"clef-flash-head-h0/1","gate":"ClefHeadGraph eager fp32 (CPU) vs the author's JointSchemaHead fp32 (CPU, same process, same module weights), on the oracle's fp32 hidden; lexical from the fp32 lm_head (bf16 values) for both","bar":{"max_abs_dlogit":0.0001},"modes":{"exact":"T, Q, O as the record has them","bucket":"T padded to the smallest bucket (512, 1024, 2048, 4096) that holds it (zero rows, key_valid 0; T > 2048 runs the key-block attention), Q to 16, O to 128 (zero rows, member 0)"},"oracle":{"path":"<lane>/oracle/records_oracle.json","sha256":"d8adaa746a5d575e654212ca78f5634af334aaa62eb53ef150d5d5925b1fa7c0","runs":214},"author_rerun_vs_oracle_max_abs_dlogit":2.1457672119140625e-06,"summary":{"exact":{"runs":214,"max_abs_dlogit_vs_author":3.6954879760742188e-06,"worst_run":"img_08/g256","max_abs_dlogit_vs_oracle":4.887580871582031e-06,"max_abs_dp_vs_author":1.5503040240449906e-07,"padding_logits_zero_all":true,"finite_all":true},"bucket":{"runs":214,"max_abs_dlogit_vs_author":3.4570693969726562e-06,"worst_run":"img_08/g256","max_abs_dlogit_vs_oracle":3.4570693969726562e-06,"max_abs_dp_vs_author":2.0775926423688418e-07,"padding_logits_zero_all":true,"finite_all":true}},"result":"PASS","script":{"gate_head.py":"1d66d14945d23d15b80866799a6287a08fcca41cf95ff56ea41e418a8c0b5e0b","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max"},"seconds":{"load":0.3477577500161715,"total":16.64892658300232},"generated_at":"2026-10-03T04:53:33+00:00","runs":[{"id":"own_t01","arm":"text","source":"own_text","tokens":436,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":436,"Q":3,"O":10},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":8.271486962030394e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.1920928955078125e-06,"max_abs_dlogit_vs_oracle":1.1920928955078125e-06,"max_abs_dp_vs_author":9.436614445412772e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t02","arm":"text","source":"own_text","tokens":599,"questions":4,"options":14,"author_rerun_vs_oracle_max_abs_dlogit":9.5367431640625e-07,"exact":{"shape":{"T":599,"Q":4,"O":14},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":1.0211054057851499e-07,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":9.897177732920426e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t03","arm":"text","source":"own_text","tokens":463,"questions":3,"options":7,"author_rerun_vs_oracle_max_abs_dlogit":7.152557373046875e-07,"exact":{"shape":{"T":463,"Q":3,"O":7},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":2.980232238769531e-07,"max_abs_dp_vs_author":7.572119642418329e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":1.197480384851346e-07,"padding_logits_zero":true,"finite":true}},{"id":"own_t04","arm":"text","source":"own_text","tokens":493,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":9.5367431640625e-07,"exact":{"shape":{"T":493,"Q":4,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.102047422906764e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.9532490825908724e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t05","arm":"text","source":"own_text","tokens":553,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":9.5367431640625e-07,"exact":{"shape":{"T":553,"Q":4,"O":13},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":9.015374524690145e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":8.635831993952792e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t06","arm":"text","source":"own_text","tokens":411,"questions":3,"options":9,"author_rerun_vs_oracle_max_abs_dlogit":9.5367431640625e-07,"exact":{"shape":{"T":411,"Q":3,"O":9},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-06,"max_abs_dp_vs_author":4.0940357978747066e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":6.091074050695866e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t07","arm":"text","source":"own_text","tokens":438,"questions":3,"options":12,"author_rerun_vs_oracle_max_abs_dlogit":9.5367431640625e-07,"exact":{"shape":{"T":438,"Q":3,"O":12},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":9.523987376525156e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":9.626579322574713e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t08","arm":"text","source":"own_text","tokens":604,"questions":4,"options":18,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":604,"Q":4,"O":18},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":8.503795656533697e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":7.620609288050417e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t09","arm":"text","source":"own_text","tokens":582,"questions":4,"options":16,"author_rerun_vs_oracle_max_abs_dlogit":1.9073486328125e-06,"exact":{"shape":{"T":582,"Q":4,"O":16},"max_abs_dlogit_vs_author":3.0994415283203125e-06,"max_abs_dlogit_vs_oracle":4.887580871582031e-06,"max_abs_dp_vs_author":1.1854339798667013e-07,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.6689300537109375e-06,"max_abs_dlogit_vs_oracle":3.4570693969726562e-06,"max_abs_dp_vs_author":1.1854339798667013e-07,"padding_logits_zero":true,"finite":true}},{"id":"own_t10","arm":"text","source":"own_text","tokens":396,"questions":3,"options":8,"author_rerun_vs_oracle_max_abs_dlogit":7.748603820800781e-07,"exact":{"shape":{"T":396,"Q":3,"O":8},"max_abs_dlogit_vs_author":6.854534149169922e-07,"max_abs_dlogit_vs_oracle":1.4603137969970703e-06,"max_abs_dp_vs_author":3.3430096246789276e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.022171761677626e-07,"padding_logits_zero":true,"finite":true}},{"id":"own_t11","arm":"text","source":"own_text","tokens":497,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":497,"Q":4,"O":11},"max_abs_dlogit_vs_author":5.960464477539062e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.450614671773877e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":5.960464477539062e-07,"max_abs_dp_vs_author":7.406701840739771e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t12","arm":"text","source":"own_text","tokens":967,"questions":8,"options":29,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":967,"Q":8,"O":29},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":9.16572077125366e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":8.315705435357756e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t13","arm":"text","source":"own_text","tokens":370,"questions":3,"options":8,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":370,"Q":3,"O":8},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.7681193104942565e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":4.37841495282143e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t14","arm":"text","source":"own_text_long","tokens":2389,"questions":4,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":2389,"Q":4,"O":15},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.6093348703959407e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":4096,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.222705210499278e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t15","arm":"text","source":"own_text_long","tokens":2603,"questions":6,"options":22,"author_rerun_vs_oracle_max_abs_dlogit":1.430511474609375e-06,"exact":{"shape":{"T":2603,"Q":6,"O":22},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":7.023494696944965e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":4096,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-06,"max_abs_dp_vs_author":8.753022395779908e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_t16","arm":"text","source":"own_text_long","tokens":2494,"questions":6,"options":19,"author_rerun_vs_oracle_max_abs_dlogit":7.152557373046875e-07,"exact":{"shape":{"T":2494,"Q":6,"O":19},"max_abs_dlogit_vs_author":1.341104507446289e-06,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.444461776387243e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":4096,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.430511474609375e-06,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":4.897203031983821e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j01","arm":"text","source":"own_json","tokens":476,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":476,"Q":3,"O":10},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":9.92951753597815e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":5.970400351884564e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j02","arm":"text","source":"own_json","tokens":475,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":475,"Q":4,"O":11},"max_abs_dlogit_vs_author":5.960464477539062e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.5531007459030945e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.3071974370766384e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j03","arm":"text","source":"own_json","tokens":585,"questions":4,"options":14,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":585,"Q":4,"O":14},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.244756034501364e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.8123511731112956e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j04","arm":"text","source":"own_json","tokens":569,"questions":4,"options":14,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":569,"Q":4,"O":14},"max_abs_dlogit_vs_author":1.430511474609375e-06,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.2152826501838945e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.430511474609375e-06,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.1711969216676721e-07,"padding_logits_zero":true,"finite":true}},{"id":"own_j05","arm":"text","source":"own_json","tokens":539,"questions":4,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":539,"Q":4,"O":15},"max_abs_dlogit_vs_author":2.1457672119140625e-06,"max_abs_dlogit_vs_oracle":2.1457672119140625e-06,"max_abs_dp_vs_author":7.837284354828e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":3.0994415283203125e-06,"max_abs_dlogit_vs_oracle":3.0994415283203125e-06,"max_abs_dp_vs_author":5.3060476457034156e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j06","arm":"text","source":"own_json","tokens":616,"questions":5,"options":17,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":616,"Q":5,"O":17},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.656510447873188e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.430511474609375e-06,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":7.651001543429459e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j07","arm":"text","source":"own_json","tokens":595,"questions":4,"options":16,"author_rerun_vs_oracle_max_abs_dlogit":1.430511474609375e-06,"exact":{"shape":{"T":595,"Q":4,"O":16},"max_abs_dlogit_vs_author":1.1920928955078125e-06,"max_abs_dlogit_vs_oracle":6.556510925292969e-07,"max_abs_dp_vs_author":6.346926226252769e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":1.7881393432617188e-06,"max_abs_dp_vs_author":6.035496447065469e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j08","arm":"text","source":"own_json","tokens":473,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":473,"Q":3,"O":10},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.8393065837638574e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.148771292124877e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j09","arm":"text","source":"own_json","tokens":573,"questions":4,"options":12,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":573,"Q":4,"O":12},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.9992681782609054e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.489069642734435e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j10","arm":"text","source":"own_json","tokens":413,"questions":3,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":413,"Q":3,"O":11},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.1546817763915556e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.1546817763915556e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j11","arm":"text","source":"own_json","tokens":463,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":463,"Q":3,"O":10},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.835757986907055e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.312850308112019e-08,"padding_logits_zero":true,"finite":true}},{"id":"own_j12","arm":"text","source":"own_json","tokens":562,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":562,"Q":4,"O":13},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.018037778168008e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":5.6948143423163344e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_01","arm":"g256","source":"own_image","tokens":430,"questions":3,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":7.152557373046875e-07,"exact":{"shape":{"T":430,"Q":3,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":7.880088215284076e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.880088215284076e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_01","arm":"g448","source":"own_image","tokens":562,"questions":3,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":9.5367431640625e-07,"exact":{"shape":{"T":562,"Q":3,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.537365272394567e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":5.960464477539062e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.0348002388305133e-07,"padding_logits_zero":true,"finite":true}},{"id":"img_01","arm":"native","source":"own_image","tokens":430,"questions":3,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":7.152557373046875e-07,"exact":{"shape":{"T":430,"Q":3,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":7.880088215284076e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.880088215284076e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_02","arm":"g256","source":"own_image","tokens":466,"questions":3,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":466,"Q":3,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.285402490673441e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.452989658762732e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_02","arm":"g448","source":"own_image","tokens":598,"questions":3,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":598,"Q":3,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.91713964387597e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.2355472822633828e-07,"padding_logits_zero":true,"finite":true}},{"id":"img_02","arm":"native","source":"own_image","tokens":702,"questions":3,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":2.1457672119140625e-06,"exact":{"shape":{"T":702,"Q":3,"O":11},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":3.066212228564069e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":1.6093254089355469e-06,"max_abs_dp_vs_author":5.702175076560678e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_03","arm":"g256","source":"own_image","tokens":495,"questions":3,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":495,"Q":3,"O":15},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.1167966096348181e-07,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.696794846965503e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_03","arm":"g448","source":"own_image","tokens":627,"questions":3,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":627,"Q":3,"O":15},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":8.218522906666692e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":7.660464218517404e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_03","arm":"native","source":"own_image","tokens":501,"questions":3,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":501,"Q":3,"O":15},"max_abs_dlogit_vs_author":5.960464477539062e-07,"max_abs_dlogit_vs_oracle":5.960464477539062e-07,"max_abs_dp_vs_author":7.866186302329936e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.7677115710389444e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_04","arm":"g256","source":"own_image","tokens":558,"questions":4,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":558,"Q":4,"O":15},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.405314899507239e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.2243728414750308e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_04","arm":"g448","source":"own_image","tokens":690,"questions":4,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":690,"Q":4,"O":15},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.992656881836922e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.598156543358158e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_04","arm":"native","source":"own_image","tokens":1262,"questions":4,"options":15,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":1262,"Q":4,"O":15},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.637518569161614e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":2048,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.16485538279332e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_05","arm":"g256","source":"own_image","tokens":464,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":464,"Q":4,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.64513471418482e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":7.789383915657311e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_05","arm":"g448","source":"own_image","tokens":596,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":596,"Q":4,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.879390442218522e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.949668131968508e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_05","arm":"native","source":"own_image","tokens":464,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":464,"Q":4,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.64513471418482e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":7.789383915657311e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_06","arm":"g256","source":"own_image","tokens":465,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":465,"Q":3,"O":10},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":7.583662819943271e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":1.62359065081219e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_06","arm":"g448","source":"own_image","tokens":597,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":597,"Q":3,"O":10},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":3.890212840751417e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.0728836059570312e-06,"max_abs_dlogit_vs_oracle":1.0728836059570312e-06,"max_abs_dp_vs_author":7.491431069928112e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_06","arm":"native","source":"own_image","tokens":471,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":471,"Q":3,"O":10},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.837491701132791e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.0728836059570312e-06,"max_abs_dlogit_vs_oracle":1.0728836059570312e-06,"max_abs_dp_vs_author":1.0045633280775235e-07,"padding_logits_zero":true,"finite":true}},{"id":"img_07","arm":"g256","source":"own_image","tokens":426,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":426,"Q":3,"O":10},"max_abs_dlogit_vs_author":4.172325134277344e-07,"max_abs_dlogit_vs_oracle":4.172325134277344e-07,"max_abs_dp_vs_author":2.9009385865697013e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.172325134277344e-07,"max_abs_dlogit_vs_oracle":4.172325134277344e-07,"max_abs_dp_vs_author":3.881266230632008e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_07","arm":"g448","source":"own_image","tokens":558,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":558,"Q":3,"O":10},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.672281276960291e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.247272332782217e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_07","arm":"native","source":"own_image","tokens":662,"questions":3,"options":10,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":662,"Q":3,"O":10},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.1667186383679251e-07,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.077689455453168e-07,"padding_logits_zero":true,"finite":true}},{"id":"img_08","arm":"g256","source":"own_image","tokens":522,"questions":4,"options":14,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":522,"Q":4,"O":14},"max_abs_dlogit_vs_author":3.6954879760742188e-06,"max_abs_dlogit_vs_oracle":3.2186508178710938e-06,"max_abs_dp_vs_author":6.932278409643722e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":3.4570693969726562e-06,"max_abs_dlogit_vs_oracle":2.9802322387695312e-06,"max_abs_dp_vs_author":5.0617171676314854e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_08","arm":"g448","source":"own_image","tokens":654,"questions":4,"options":14,"author_rerun_vs_oracle_max_abs_dlogit":9.5367431640625e-07,"exact":{"shape":{"T":654,"Q":4,"O":14},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":5.364418029785156e-07,"max_abs_dp_vs_author":6.720451972697106e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.773806561164662e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_08","arm":"native","source":"own_image","tokens":1226,"questions":4,"options":14,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":1226,"Q":4,"O":14},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":6.969107702037292e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":2048,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-06,"max_abs_dp_vs_author":6.969107702037292e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_09","arm":"g256","source":"own_image","tokens":460,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":5.960464477539062e-07,"exact":{"shape":{"T":460,"Q":4,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.682378922955422e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.3063394789276686e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_09","arm":"g448","source":"own_image","tokens":592,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":2.384185791015625e-07,"exact":{"shape":{"T":592,"Q":4,"O":11},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":5.593669261116929e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.593669261116929e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_09","arm":"native","source":"own_image","tokens":696,"questions":4,"options":11,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":696,"Q":4,"O":11},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.641915563041721e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.544163640345488e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_10","arm":"g256","source":"own_image","tokens":405,"questions":3,"options":9,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":405,"Q":3,"O":9},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.71714655511957e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":5.591296425855319e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_10","arm":"g448","source":"own_image","tokens":537,"questions":3,"options":9,"author_rerun_vs_oracle_max_abs_dlogit":5.960464477539062e-07,"exact":{"shape":{"T":537,"Q":3,"O":9},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.568523823660463e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.748603820800781e-07,"max_abs_dp_vs_author":1.8410270974378307e-08,"padding_logits_zero":true,"finite":true}},{"id":"img_10","arm":"native","source":"own_image","tokens":405,"questions":3,"options":9,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":405,"Q":3,"O":9},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.247079776753537e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.637953743280576e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_01","arm":"g256","source":"cc0_photo","tokens":556,"questions":4,"options":16,"author_rerun_vs_oracle_max_abs_dlogit":2.384185791015625e-07,"exact":{"shape":{"T":556,"Q":4,"O":16},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":1.5503040240449906e-07,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":1.2006160676758526e-07,"padding_logits_zero":true,"finite":true}},{"id":"photo_01","arm":"g448","source":"cc0_photo","tokens":688,"questions":4,"options":16,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":688,"Q":4,"O":16},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":7.53441474765637e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":8.847563992553376e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_01","arm":"native","source":"cc0_photo","tokens":1692,"questions":4,"options":16,"author_rerun_vs_oracle_max_abs_dlogit":5.960464477539062e-07,"exact":{"shape":{"T":1692,"Q":4,"O":16},"max_abs_dlogit_vs_author":8.344650268554688e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":7.175241867329873e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":2048,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":8.473910551032304e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_02","arm":"g256","source":"cc0_photo","tokens":499,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":499,"Q":4,"O":13},"max_abs_dlogit_vs_author":5.364418029785156e-07,"max_abs_dlogit_vs_oracle":5.960464477539062e-07,"max_abs_dp_vs_author":7.700331183801978e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":5.915365219877344e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_02","arm":"g448","source":"cc0_photo","tokens":631,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":1.430511474609375e-06,"exact":{"shape":{"T":631,"Q":4,"O":13},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":5.960464477539062e-07,"max_abs_dp_vs_author":5.6512926116525364e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.624829897161622e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_02","arm":"native","source":"cc0_photo","tokens":1035,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":1035,"Q":4,"O":13},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.019906073628789e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":2048,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.930911855551614e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_03","arm":"g256","source":"cc0_photo","tokens":505,"questions":4,"options":12,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":505,"Q":4,"O":12},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":4.382955620663154e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.279649328837621e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_03","arm":"g448","source":"cc0_photo","tokens":637,"questions":4,"options":12,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":637,"Q":4,"O":12},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.216798431601944e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.798171552941824e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_03","arm":"native","source":"cc0_photo","tokens":1041,"questions":4,"options":12,"author_rerun_vs_oracle_max_abs_dlogit":7.152557373046875e-07,"exact":{"shape":{"T":1041,"Q":4,"O":12},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.58307071182773e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":2048,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.725679330659261e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_04","arm":"g256","source":"cc0_photo","tokens":502,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":502,"Q":4,"O":13},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.079120158844262e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.132318060483641e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_04","arm":"g448","source":"cc0_photo","tokens":634,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":634,"Q":4,"O":13},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":7.538935664630486e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":1024,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":9.07020383245083e-08,"padding_logits_zero":true,"finite":true}},{"id":"photo_04","arm":"native","source":"cc0_photo","tokens":1038,"questions":4,"options":13,"author_rerun_vs_oracle_max_abs_dlogit":4.76837158203125e-07,"exact":{"shape":{"T":1038,"Q":4,"O":13},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.764787320254982e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":2048,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.0005139584641256e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_a3f18f3a63d45345942b","arm":"text","source":"semif_authored144","tokens":185,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":185,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.6441229145367515e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.6441229145367515e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_f40beba9088c8db8bbd6","arm":"text","source":"semif_authored144","tokens":183,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":183,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":9.614895768450538e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.4136270121433157e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_0b43ea8e24e74d621f1b","arm":"text","source":"semif_authored144","tokens":177,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":177,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.382692659315012e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.215242123071448e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_942ec65dac0bfb9874f9","arm":"text","source":"semif_authored144","tokens":177,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":177,"Q":1,"O":3},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":3.5762786865234375e-07,"max_abs_dp_vs_author":5.5010687199930075e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":7.411705750526298e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_e0c140e9222d1bce5647","arm":"text","source":"semif_authored144","tokens":189,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":189,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.2704425074450132e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.0375446690978904e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_8e8c3804a3c15ebb31e3","arm":"text","source":"semif_authored144","tokens":186,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":186,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":1.748130386280966e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.9711533428011805e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_56c419b12cd37cbe790c","arm":"text","source":"semif_authored144","tokens":188,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":188,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.1184302328021145e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.0842787213103406e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_75b470586834df8c568d","arm":"text","source":"semif_authored144","tokens":181,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":181,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.5663302805643013e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.5663302805643013e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_834f3b3e10cc61b33d25","arm":"text","source":"semif_authored144","tokens":175,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":175,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.867691870748871e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.041497568887166e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_76c7a79b1ff2363a60c5","arm":"text","source":"semif_authored144","tokens":175,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":175,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.980232238769531e-07,"max_abs_dlogit_vs_oracle":2.980232238769531e-07,"max_abs_dp_vs_author":4.8091174931208513e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.7881393432617188e-07,"max_abs_dlogit_vs_oracle":1.7881393432617188e-07,"max_abs_dp_vs_author":4.5301454476565084e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_36c71bb649b7d5eccb1f","arm":"text","source":"semif_authored144","tokens":174,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":174,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.279082032101257e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.576109006812715e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_eac5121a4a1bf391a08c","arm":"text","source":"semif_authored144","tokens":173,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":173,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":9.113169474606053e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":8.77876914495701e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_44574c44e243ee1b2f52","arm":"text","source":"semif_authored144","tokens":186,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":186,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.7769729687699964e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.9199990758677075e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_2608370d01783735c3ea","arm":"text","source":"semif_authored144","tokens":186,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":186,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":5.7191243962861904e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.1151444412327294e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_ec94ef909f7918695ab3","arm":"text","source":"semif_authored144","tokens":180,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":180,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.713524855340154e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.9438262751091404e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_03d919fd0e9b3a7cd1e6","arm":"text","source":"semif_authored144","tokens":175,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":175,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":3.597744213656995e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.9628730180995007e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_86a15618ba4cb90a307e","arm":"text","source":"semif_authored144","tokens":179,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":179,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.16237784325968e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.31048152621716e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_3dd9c3ac98f8ddf7bdd3","arm":"text","source":"semif_authored144","tokens":179,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":179,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":4.8724174361858275e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":4.8724174361858275e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_36273dd4777876a26338","arm":"text","source":"semif_authored144","tokens":175,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":175,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.5916559898142566e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.597222320017579e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_07ee70cfe4b01330ce89","arm":"text","source":"semif_authored144","tokens":175,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":175,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.987583103131186e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":9.314189131082173e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_51ceab3ce622c7c759f1","arm":"text","source":"semif_authored144","tokens":191,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":191,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.654383725152586e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.9216741137538804e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b8091be929b1e489fcc4","arm":"text","source":"semif_authored144","tokens":191,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":191,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.343237638455456e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.696331730973501e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_c89b86e3f55123bfa506","arm":"text","source":"semif_authored144","tokens":182,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":182,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":4.667786390744055e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.5631451728311845e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_3b0958cabc339005cf66","arm":"text","source":"semif_authored144","tokens":178,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":178,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.872851311343851e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.872851311343851e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_f2b4ec4930322fe45118","arm":"text","source":"semif_authored144","tokens":193,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":193,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.0321726003358354e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.454544572623064e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_61ac20b07a5189231e15","arm":"text","source":"semif_authored144","tokens":199,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":199,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.97410796387976e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.716424227892759e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4b3d794c004566610181","arm":"text","source":"semif_authored144","tokens":184,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":184,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.0316773435548043e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.859638371214949e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_b7c4e62d07986286fce0","arm":"text","source":"semif_authored144","tokens":189,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":189,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.6443013801124025e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":5.8096893518211346e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_9473603b44af63beab79","arm":"text","source":"semif_authored144","tokens":190,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":190,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.490913001007016e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.5782904367055437e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_d41072dc35d8cd14c056","arm":"text","source":"semif_authored144","tokens":195,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":195,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.355424975179602e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.922045514556885e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_33d6e5da58f0fee2490d","arm":"text","source":"semif_authored144","tokens":190,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":190,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":2.2242054198429173e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.067268434213787e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_823bf736b92e9d8fa66d","arm":"text","source":"semif_authored144","tokens":190,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":190,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.495420578842779e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.495420578842779e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_8a4c3de28be95c105ca1","arm":"text","source":"semif_authored144","tokens":189,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":189,"Q":1,"O":3},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":3.1516289716870816e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":6.628236537409293e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_ea7ec66a5e8eac886548","arm":"text","source":"semif_authored144","tokens":192,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":192,"Q":1,"O":3},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":8.68865522862805e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":6.694983434307744e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_25172cd6ddc49ab0ca70","arm":"text","source":"semif_authored144","tokens":183,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":183,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":6.662984020078255e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.759102282900443e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_fb73a85d386cf59b0c51","arm":"text","source":"semif_authored144","tokens":185,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":185,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.136647158674677e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.430511474609375e-06,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":1.6885620424123005e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_a02fd582d2881cc462b2","arm":"text","source":"semif_authored144","tokens":195,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":195,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.8689610898655644e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.430511474609375e-06,"max_abs_dlogit_vs_oracle":1.430511474609375e-06,"max_abs_dp_vs_author":5.730941798898215e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_19f37d5a97034bd7d04f","arm":"text","source":"semif_authored144","tokens":195,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":195,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.3172629969443461e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.474099594819811e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_fda0ca94652c6d739ecf","arm":"text","source":"semif_authored144","tokens":194,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":194,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":5.855171791857572e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.9628259795290148e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_d0e36c4cb10a29b3191c","arm":"text","source":"semif_authored144","tokens":193,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":193,"Q":1,"O":3},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":3.5762786865234375e-07,"max_abs_dp_vs_author":4.408956566503974e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":5.960464477539062e-07,"max_abs_dlogit_vs_oracle":5.960464477539062e-07,"max_abs_dp_vs_author":8.890781533210657e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_a0a39ace7e5337d038a7","arm":"text","source":"semif_authored144","tokens":197,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":197,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.244060178963508e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.244060178963508e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_eeae283841ce75dd25bf","arm":"text","source":"semif_authored144","tokens":197,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":197,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.209827421051358e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.5074243409074484e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_00bb4cc043f7d2b288d4","arm":"text","source":"semif_authored144","tokens":189,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":189,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":5.5682175092997e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.575541366959925e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_1a81ead69e93e5d0c86a","arm":"text","source":"semif_authored144","tokens":196,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":196,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.335289039396727e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.335289039396727e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4444dad25e438f128673","arm":"text","source":"semif_authored144","tokens":190,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":190,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.348810205862975e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.4814786003801714e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_3226202ba1ada2570e9f","arm":"text","source":"semif_authored144","tokens":193,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":193,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":1.9607434086077458e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":1.9607434086077458e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_3f783eec5c90812479bc","arm":"text","source":"semif_authored144","tokens":190,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":190,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.3179900615334361e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.5531392333633107e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_ffee861474a9665a093a","arm":"text","source":"semif_authored144","tokens":191,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":191,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":5.137720304748683e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.0775926423688418e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_8c992e17e4fb54b044f5","arm":"text","source":"semif_authored144","tokens":178,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":178,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":5.2363798630494784e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.287243509677353e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_695dd61195ca025c4f4c","arm":"text","source":"semif_authored144","tokens":179,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":179,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.0574476233825578e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.413150218012362e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_e6155431ab7b0f8170d4","arm":"text","source":"semif_authored144","tokens":170,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":170,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":4.340657389079183e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.6502560186189044e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b49ecda75086d8c68d2a","arm":"text","source":"semif_authored144","tokens":170,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":170,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.0728836059570312e-06,"max_abs_dlogit_vs_oracle":1.0728836059570312e-06,"max_abs_dp_vs_author":1.2737985566335652e-07,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":5.97063013474397e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_587ced261c6092cd1325","arm":"text","source":"semif_authored144","tokens":186,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":186,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.369472162036759e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.948098098976004e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_533d4423d311de82b27d","arm":"text","source":"semif_authored144","tokens":189,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":189,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.494450983078991e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":4.2038126091270556e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_2bf540d515a51755b25c","arm":"text","source":"semif_authored144","tokens":178,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":178,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":7.364602341158877e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":8.131024815255516e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_fdb9fc214aea2710763a","arm":"text","source":"semif_authored144","tokens":168,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":168,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.412200106327589e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.400021004156372e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_f6d68126b43f8c9ed183","arm":"text","source":"semif_authored144","tokens":181,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":181,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.9446805055167715e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.6865436875134208e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_44bae0c8a1615a096c43","arm":"text","source":"semif_authored144","tokens":184,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":184,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.105228240154247e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":5.111841172578124e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_2175379f4c69e6207626","arm":"text","source":"semif_authored144","tokens":175,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":175,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.8608772163762897e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":2.65382733566355e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_def9fe59d262357a673a","arm":"text","source":"semif_authored144","tokens":170,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":170,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":4.2839930047833263e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":4.2839930047833263e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_712f6ffb763eef7c9a12","arm":"text","source":"semif_authored144","tokens":181,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":181,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.268533054270108e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.168110579958693e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_e1d610bd14e3d16c09b9","arm":"text","source":"semif_authored144","tokens":181,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":181,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.4199603027882404e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.0733168504836215e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_22ec57822db50ec121cf","arm":"text","source":"semif_authored144","tokens":171,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":171,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.591316670245817e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.902554588490318e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_3e75f2b84e8e08e3d3f2","arm":"text","source":"semif_authored144","tokens":170,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":170,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":5.4181225506155783e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":3.5762786865234375e-07,"max_abs_dp_vs_author":8.087341218931954e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_ebd74aa05324a1556288","arm":"text","source":"semif_authored144","tokens":186,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":186,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":6.759774417997022e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.245661442529695e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_d7e84e4245a29817da1f","arm":"text","source":"semif_authored144","tokens":184,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":184,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.76617552991732e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.113190885783439e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_f41ae66b3c09630c7a42","arm":"text","source":"semif_authored144","tokens":178,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":178,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":5.775404410268692e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.814419778982227e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_82c2cd97c160ad27951b","arm":"text","source":"semif_authored144","tokens":171,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":171,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.1858837056788474e-07,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.261892038228467e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_7ada97b6f99f4788e5c7","arm":"text","source":"semif_authored144","tokens":184,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":184,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.03674523305375e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.134313175667813e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4d7a003342db5b0a17e3","arm":"text","source":"semif_authored144","tokens":187,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":187,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.1230569410200508e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.0845980669138555e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_19d3d8d0bf773e88be7b","arm":"text","source":"semif_authored144","tokens":172,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":172,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.848357236854952e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.6926705127580135e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_d6de731cca52359b1c87","arm":"text","source":"semif_authored144","tokens":170,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":170,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.7179414708331535e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":3.2011179124102895e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b45c63fb92729af27f05","arm":"text","source":"semif_authored144","tokens":197,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":197,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":6.87817315325745e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.357729727593295e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_46b7029b9a704138b77a","arm":"text","source":"semif_authored144","tokens":195,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":195,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.9415306473646865e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":3.03626370712351e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_caf773b23934fa7362a1","arm":"text","source":"semif_authored144","tokens":190,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":190,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":4.548688814498547e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.3796457138125504e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_f46f392ef9e9e9df564b","arm":"text","source":"semif_authored144","tokens":188,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":188,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.574756915624704e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.092797493779557e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_005dcf6d2c0f21d64796","arm":"text","source":"semif_authored144","tokens":209,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":209,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.8599307033694288e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.053299773789984e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_edb708bd0fe5fd67d6ee","arm":"text","source":"semif_authored144","tokens":210,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":210,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":5.6777193169210705e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.9612341346261246e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_97a35499bd8717f3e786","arm":"text","source":"semif_authored144","tokens":203,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":203,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.4271509246887035e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.3960822687074668e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_31eb4533d2c89959ec46","arm":"text","source":"semif_authored144","tokens":202,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":202,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":5.4402658267171944e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.5376801987265765e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_0aa24a6701ac9c56c4a0","arm":"text","source":"semif_authored144","tokens":194,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":194,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.4613125759742616e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.265275499459875e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_796d5c0da6eaadae6a3f","arm":"text","source":"semif_authored144","tokens":188,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":188,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.260931575293057e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":5.960464477539062e-07,"max_abs_dlogit_vs_oracle":5.960464477539062e-07,"max_abs_dp_vs_author":2.7332134888702342e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_599b3092b6fd18312d23","arm":"text","source":"semif_authored144","tokens":187,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":187,"Q":1,"O":3},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":3.5762786865234375e-07,"max_abs_dp_vs_author":2.5384101043357532e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.034142358528726e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_960a4bcd27363e6db4b9","arm":"text","source":"semif_authored144","tokens":189,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":189,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":5.948976555814056e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":3.6126727609442355e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_c6a411e7f2bbdf377173","arm":"text","source":"semif_authored144","tokens":207,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":207,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":9.458254535132937e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.442214901947736e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_e332aa927d0c2ed044aa","arm":"text","source":"semif_authored144","tokens":208,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":208,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":6.293140719648704e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.0567697950047972e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_a0128e551d095f3449f8","arm":"text","source":"semif_authored144","tokens":183,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":183,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.2693777190774256e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.3964076490432547e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_46d6a0731bd288c370f0","arm":"text","source":"semif_authored144","tokens":182,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":182,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.1222943059082695e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.829728278947897e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_21ca7a8b970e36b8e20f","arm":"text","source":"semif_authored144","tokens":214,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":214,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":1.6743591846335448e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.7068698716401514e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4aa83af95bb69e90bc2d","arm":"text","source":"semif_authored144","tokens":212,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":212,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":5.1550730573168835e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.4691326195334113e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_cdfa9eed51fafcf09b72","arm":"text","source":"semif_authored144","tokens":197,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":197,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":5.8481323561565546e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":3.5762786865234375e-07,"max_abs_dp_vs_author":2.941774523979035e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_d51b05bfb9d6d7f09331","arm":"text","source":"semif_authored144","tokens":203,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":203,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.283644762179904e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.0614328421754138e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_fdd902cc6e65fad7cb4f","arm":"text","source":"semif_authored144","tokens":201,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":201,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.460040382512574e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.03661111325515e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_42384af9ae56beb37ff5","arm":"text","source":"semif_authored144","tokens":201,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":201,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":2.1738339461840894e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":1.3443459345729991e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_cdf7f428857e1b841cb3","arm":"text","source":"semif_authored144","tokens":194,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":194,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.9101969429598e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.9101969429598e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_774f7f9245f83c796d44","arm":"text","source":"semif_authored144","tokens":180,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":180,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":1.735591803608827e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":9.152530955347515e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_92ec2ffa3dd46ada1335","arm":"text","source":"semif_authored144","tokens":204,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":204,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.2681376783684204e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":1.5717850070096073e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_6149a17bc154f9c5b4a0","arm":"text","source":"semif_authored144","tokens":200,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":200,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.119035318910335e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.119035318910335e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_be866e1dbec5d42d9423","arm":"text","source":"semif_authored144","tokens":201,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":201,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":1.9837224196628256e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":3.5762786865234375e-07,"max_abs_dp_vs_author":7.982699362640844e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_97d2ed2591fc7638806c","arm":"text","source":"semif_authored144","tokens":199,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":199,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.8716577549789974e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.8716577549789974e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4533dd91267b4d815690","arm":"text","source":"semif_authored144","tokens":205,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":205,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.1709256493049907e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.3780959090835552e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_d880e65fef5e2993fdd0","arm":"text","source":"semif_authored144","tokens":202,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":202,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.8884816921935794e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":3.5762786865234375e-07,"max_abs_dlogit_vs_oracle":3.5762786865234375e-07,"max_abs_dp_vs_author":3.9266310203345256e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_90494c76078ffc3c717a","arm":"text","source":"semif_authored144","tokens":202,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":202,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":5.08867364912291e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.88122755046183e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_9ac5cc82c4126c91174a","arm":"text","source":"semif_authored144","tokens":201,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":201,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.473014019572123e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":5.491678112257148e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_386e10b1c1ce906473a4","arm":"text","source":"semif_authored144","tokens":205,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":205,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":2.4829264089198944e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.5428168903829373e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4da6aedc1139224eeff2","arm":"text","source":"semif_authored144","tokens":208,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":208,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.612469200499959e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.596737517876704e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_a22dab8ba073ff6cdb8d","arm":"text","source":"semif_authored144","tokens":201,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":201,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.7405793301404913e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.7900561416771552e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_42c54fa2ad45743e24d8","arm":"text","source":"semif_authored144","tokens":188,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":188,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.7305185001935683e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.649086489585727e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_f3d13cf184f916677def","arm":"text","source":"semif_authored144","tokens":208,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":208,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":9.665947386938711e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.497506421787591e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_1105577da4c8dad4609d","arm":"text","source":"semif_authored144","tokens":205,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":205,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.062122294616998e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":7.152557373046875e-07,"max_abs_dlogit_vs_oracle":7.152557373046875e-07,"max_abs_dp_vs_author":6.764459481445328e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_0ebd299739302d8b057b","arm":"text","source":"semif_authored144","tokens":201,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":201,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.324034542589004e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":5.3015489775987135e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_eafc22c8c40df3932a8e","arm":"text","source":"semif_authored144","tokens":189,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":189,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":9.692429886465348e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.56412476112855e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_0aa1447680e4f25862bb","arm":"text","source":"semif_authored144","tokens":208,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":208,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":6.05596710140377e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":5.651636814096861e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_980e76145c925747220c","arm":"text","source":"semif_authored144","tokens":209,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":209,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.8525084150586864e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":6.720720954755954e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4aee37a6a155747766aa","arm":"text","source":"semif_authored144","tokens":206,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":206,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":1.0467416489134962e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.474217297709913e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_6db17cc22bd656d78558","arm":"text","source":"semif_authored144","tokens":199,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":199,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.7988709167844377e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.85255726606826e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_ef3e0f8e657bf59f8a29","arm":"text","source":"semif_authored144","tokens":200,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":200,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":8.105265192348554e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":8.05682306426192e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_f756e1cb04e11766341e","arm":"text","source":"semif_authored144","tokens":206,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":206,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":3.706519291490906e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.295241702924947e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_d167e51bb1f499099645","arm":"text","source":"semif_authored144","tokens":186,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":186,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":7.797812806575877e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.0246949749825873e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_674ca4e9346216255466","arm":"text","source":"semif_authored144","tokens":176,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":176,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.511161900642378e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.4041153240107462e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_1df851a202994a07f123","arm":"text","source":"semif_authored144","tokens":194,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":194,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":2.260093812100905e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.064615471450651e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b5e6788d95b8395d2888","arm":"text","source":"semif_authored144","tokens":193,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":193,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":4.0836434900271e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.880173514771769e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_32f36cd0fbbfb3604586","arm":"text","source":"semif_authored144","tokens":183,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":183,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.89394421151934e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":2.5428371852598275e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_843185706adff0c60f8b","arm":"text","source":"semif_authored144","tokens":184,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":184,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":1.570200597322824e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.8010361618747694e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b531dd603bf080dab820","arm":"text","source":"semif_authored144","tokens":199,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":199,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.890518074367577e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":5.960464477539063e-08,"max_abs_dlogit_vs_oracle":5.960464477539063e-08,"max_abs_dp_vs_author":2.7550454806402058e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_be2dd67bc733c1213bef","arm":"text","source":"semif_authored144","tokens":202,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":202,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.9853599098063057e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.3367393192353916e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b8e2750307df9761e716","arm":"text","source":"semif_authored144","tokens":196,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":196,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":7.617809238968931e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":7.617809238968931e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_5d31b4bf2d0fc193f53b","arm":"text","source":"semif_authored144","tokens":201,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":201,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":3.6319693696995614e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.1685213291711136e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_a9230edfc6ec9e524af0","arm":"text","source":"semif_authored144","tokens":185,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":185,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.7709546717981084e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":1.0011961038003392e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_c100e84585149a04be49","arm":"text","source":"semif_authored144","tokens":191,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":191,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":6.112526262924689e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":9.190213067986064e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_579beed184ffc1104aa9","arm":"text","source":"semif_authored144","tokens":182,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":182,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.509182078254014e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.610052528608577e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_c4e9bd463b6fde5be743","arm":"text","source":"semif_authored144","tokens":170,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":170,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.7881393432617188e-07,"max_abs_dlogit_vs_oracle":1.7881393432617188e-07,"max_abs_dp_vs_author":6.4653293170569e-09,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.7881393432617188e-07,"max_abs_dlogit_vs_oracle":1.7881393432617188e-07,"max_abs_dp_vs_author":1.137680294416299e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b863998d4be2721a5dfd","arm":"text","source":"semif_authored144","tokens":205,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":205,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":4.172062717522351e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":1.528758509783401e-07,"padding_logits_zero":true,"finite":true}},{"id":"semif_37cebc3792e98049439f","arm":"text","source":"semif_authored144","tokens":202,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":202,"Q":1,"O":3},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.8573267973897032e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":1.7881393432617188e-07,"max_abs_dlogit_vs_oracle":1.7881393432617188e-07,"max_abs_dp_vs_author":1.642120772693545e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_64d63f86e328aff1481c","arm":"text","source":"semif_authored144","tokens":210,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":210,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":1.0332650957067813e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":1.8640217137183557e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_6d64a8459f64eac7f4cc","arm":"text","source":"semif_authored144","tokens":207,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":207,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":1.226674439802622e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":8.468179213405591e-09,"padding_logits_zero":true,"finite":true}},{"id":"semif_2708193212d8a4a523c7","arm":"text","source":"semif_authored144","tokens":207,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":207,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":4.9627868037305234e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":9.413730905905027e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_5a37d1dfcad20810a76a","arm":"text","source":"semif_authored144","tokens":196,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":196,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.898595089095295e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":7.898595089095295e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_acacd8b46acd25cb954b","arm":"text","source":"semif_authored144","tokens":206,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":206,"Q":1,"O":3},"max_abs_dlogit_vs_author":0.0,"max_abs_dlogit_vs_oracle":0.0,"max_abs_dp_vs_author":3.237285717272442e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":5.062479935258324e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_4089c199b8b796ec3ed8","arm":"text","source":"semif_authored144","tokens":205,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":205,"Q":1,"O":3},"max_abs_dlogit_vs_author":1.1920928955078125e-07,"max_abs_dlogit_vs_oracle":1.1920928955078125e-07,"max_abs_dp_vs_author":4.689597687956848e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.4055122216946074e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_02f2b95a5c5a4777a45c","arm":"text","source":"semif_authored144","tokens":191,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":191,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":3.349284782849793e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":3.3834463786242e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_ebea4681e3dcc7d92ca2","arm":"text","source":"semif_authored144","tokens":196,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":196,"Q":1,"O":3},"max_abs_dlogit_vs_author":4.76837158203125e-07,"max_abs_dlogit_vs_oracle":4.76837158203125e-07,"max_abs_dp_vs_author":4.995994795642389e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":3.3820622566405056e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_b8c5b9b285cbdddc9f8d","arm":"text","source":"semif_authored144","tokens":187,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":187,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.667919207777203e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":9.5367431640625e-07,"max_abs_dlogit_vs_oracle":9.5367431640625e-07,"max_abs_dp_vs_author":2.0854703630668325e-08,"padding_logits_zero":true,"finite":true}},{"id":"semif_7ed75d854d99b3a81916","arm":"text","source":"semif_authored144","tokens":185,"questions":1,"options":3,"author_rerun_vs_oracle_max_abs_dlogit":0.0,"exact":{"shape":{"T":185,"Q":1,"O":3},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.5940377304523032e-08,"padding_logits_zero":true,"finite":true},"bucket":{"shape":{"T":512,"Q":16,"O":128},"max_abs_dlogit_vs_author":2.384185791015625e-07,"max_abs_dlogit_vs_oracle":2.384185791015625e-07,"max_abs_dp_vs_author":2.5290081154261124e-08,"padding_logits_zero":true,"finite":true}}]},"h1_fixture":{"schema":"clef-flash-head-h1/1","gate":"head graph alone on the oracle's fp32 hidden (lexical from the fp16 table)","bar":{"max_abs_dp":0.001,"argmax":"every question"},"checks":{"all_runs":true,"argmax_every_question":true,"max_abs_dp":true,"finite_and_padding_zero":true,"rerun_bit_equal":true},"result":"PASS","summary":{"runs":214,"questions":405,"argmax_equal":405,"questions_non_near_tie":401,"argmax_equal_non_near_tie":401,"near_tie_questions":4,"argmax_equal_near_tie":4,"max_abs_dp":1.7881393432617188e-07,"mean_of_run_mean_abs_dp":2.0149952578016223e-08,"max_abs_dlogit":2.9802322387695312e-06,"worst_run":"own_t08/text"},"by_source_arm":{"cc0_photo/g256":{"runs":4,"questions":16,"argmax_equal":16,"questions_non_near_tie":15,"argmax_equal_non_near_tie":15,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.492933220922565e-08,"max_abs_dlogit":1.1324882507324219e-06,"worst_run":"photo_01/g256"},"cc0_photo/g448":{"runs":4,"questions":16,"argmax_equal":16,"questions_non_near_tie":16,"argmax_equal_non_near_tie":16,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.865704782689229e-08,"max_abs_dlogit":9.5367431640625e-07,"worst_run":"photo_03/g448"},"cc0_photo/native":{"runs":4,"questions":16,"argmax_equal":16,"questions_non_near_tie":16,"argmax_equal_non_near_tie":16,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.7881393432617188e-07,"mean_of_run_mean_abs_dp":1.796321997132439e-08,"max_abs_dlogit":9.5367431640625e-07,"worst_run":"photo_01/native"},"own_image/g256":{"runs":10,"questions":34,"argmax_equal":34,"questions_non_near_tie":33,"argmax_equal_non_near_tie":33,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.3936966829657726e-08,"max_abs_dlogit":1.430511474609375e-06,"worst_run":"img_09/g256"},"own_image/g448":{"runs":10,"questions":34,"argmax_equal":34,"questions_non_near_tie":33,"argmax_equal_non_near_tie":33,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.668258775751312e-08,"max_abs_dlogit":1.7881393432617188e-06,"worst_run":"img_01/g448"},"own_image/native":{"runs":10,"questions":34,"argmax_equal":34,"questions_non_near_tie":33,"argmax_equal_non_near_tie":33,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.746614182752276e-08,"max_abs_dlogit":2.0265579223632812e-06,"worst_run":"img_02/native"},"own_json/text":{"runs":12,"questions":45,"argmax_equal":45,"questions_non_near_tie":45,"argmax_equal_non_near_tie":45,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.4901161193847656e-07,"mean_of_run_mean_abs_dp":1.9658799379438202e-08,"max_abs_dlogit":1.6689300537109375e-06,"worst_run":"own_j09/text"},"own_text/text":{"runs":13,"questions":50,"argmax_equal":50,"questions_non_near_tie":50,"argmax_equal_non_near_tie":50,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.7881393432617188e-07,"mean_of_run_mean_abs_dp":1.816149572841353e-08,"max_abs_dlogit":2.9802322387695312e-06,"worst_run":"own_t08/text"},"own_text_long/text":{"runs":3,"questions":16,"argmax_equal":16,"questions_non_near_tie":16,"argmax_equal_non_near_tie":16,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":7.3076515462980315e-09,"max_abs_dlogit":9.5367431640625e-07,"worst_run":"own_t16/text"},"semif_authored144/text":{"runs":144,"questions":144,"argmax_equal":144,"questions_non_near_tie":144,"argmax_equal_non_near_tie":144,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.7881393432617188e-07,"mean_of_run_mean_abs_dp":2.1743795110119713e-08,"max_abs_dlogit":1.6689300537109375e-06,"worst_run":"semif_ffee861474a9665a093a/text"}},"head_bundle":{"dir":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","metadata_sha256":"631219aafcdb56ef1f008e5476f0e7a282933abdc51b4835132676d97124ab20","aimodelc":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_metadata":{"graph":"conversion/clef_flash/clef_head.py ClefHeadGraph: the author's JointSchemaHead for one record, tensor inputs only","shape":"bucket","functions":["t512","t1024","t2048","t4096"],"buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"dims":null,"host_pad":"bucket","host_pad_rule":"T to the smallest bucket >= T (512, 1024, 2048, 4096), Q to 16, O to 128; padding rows / columns zero, their valid flags 0","inputs":{"hidden":"[T, 4096] f32","key_valid":"[T] f32","q_avg":"[Q, T] f32","o_avg":"[O, T] f32","g_avg":"[1, T] f32","lexical":"[O, 4096] f32","member":"[Q, O] f32","type_ids":"[Q] i32","q_valid":"[Q] f32","o_valid":"[O] f32"},"input_order":["hidden","key_valid","q_avg","o_avg","g_avg","lexical","member","type_ids","q_valid","o_valid"],"outputs":{"logits":"[O] f32; padding options 0"},"weights":"fp16w32","softmax":"host: per question, fp32, over that question's options","lexical":"host: lm_head_fp16.bin rows of each option span's ids, averaged (clef_head.lexical_rows)","source":{"hf_id":"Cloudflare/clef-flash","revision":"17f0b0ad64efb65d273590632833508766b2aae6","joint_schema_model_sha256":"0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3","joint_head_safetensors_sha256":"19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0","joint_head_config_sha256":"77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0"}}},"descriptors":{"t512":{"function":"t512","inputs":{"hidden":[[512,4096],"float32"],"key_valid":[[512],"float32"],"q_avg":[[16,512],"float32"],"o_avg":[[128,512],"float32"],"g_avg":[[1,512],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t1024":{"function":"t1024","inputs":{"hidden":[[1024,4096],"float32"],"key_valid":[[1024],"float32"],"q_avg":[[16,1024],"float32"],"o_avg":[[128,1024],"float32"],"g_avg":[[1,1024],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t2048":{"function":"t2048","inputs":{"hidden":[[2048,4096],"float32"],"key_valid":[[2048],"float32"],"q_avg":[[16,2048],"float32"],"o_avg":[[128,2048],"float32"],"g_avg":[[1,2048],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t4096":{"function":"t4096","inputs":{"hidden":[[4096,4096],"float32"],"key_valid":[[4096],"float32"],"q_avg":[[16,4096],"float32"],"o_avg":[[128,4096],"float32"],"g_avg":[[1,4096],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}}},"descriptors_same_every_process":true,"oracle":{"path":"<lane>/oracle/records_oracle.json","sha256":"d8adaa746a5d575e654212ca78f5634af334aaa62eb53ef150d5d5925b1fa7c0","runs":214},"table":{"path":"<lane>/exports/host/lm_head_fp16.bin","json":"<lane>/exports/host/lm_head_fp16.json"},"near_ties":[{"id":"img_01","arm":"g256","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"max_abs_dp":2.9802322387695312e-08,"probs":[0.4905681312084198,0.48838353157043457,0.021048370748758316],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_01","arm":"native","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"max_abs_dp":2.9802322387695312e-08,"probs":[0.4905681312084198,0.48838353157043457,0.021048370748758316],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_06","arm":"g448","question_id":"in_red_zone","oracle_top2_margin":0.017518073320388794,"argmax_equal":true,"max_abs_dp":8.940696716308594e-08,"probs":[0.5087589621543884,0.4912410378456116],"probs_oracle":[0.5087590217590332,0.4912409484386444]},{"id":"photo_01","arm":"g256","question_id":"kinds","oracle_top2_margin":0.00908571481704712,"argmax_equal":true,"max_abs_dp":6.705522537231445e-08,"probs":[0.023699799552559853,0.11542533338069916,0.38127321004867554,0.39035898447036743,0.08924271166324615],"probs_oracle":[0.023699797689914703,0.11542528122663498,0.3812732696533203,0.39035898447036743,0.08924264460802078]}],"timing":{"contended":true,"by_T":{"T<=300":{"runs":144,"median_tokens":190.0,"median_padded_T":512.0,"first_call_ms_median":6.482500000856817,"second_call_ms_median":6.033812489476986,"host_ms_median":0.2844170085154474},"300<T<=700":{"runs":59,"median_tokens":522.0,"median_padded_T":1024.0,"first_call_ms_median":8.121207996737212,"second_call_ms_median":7.683291973080486,"host_ms_median":0.676124996971339},"700<T<=1500":{"runs":7,"median_tokens":1038.0,"median_padded_T":2048.0,"first_call_ms_median":11.848875001305714,"second_call_ms_median":11.100542003987357,"host_ms_median":1.2227910046931356},"T>1500":{"runs":4,"median_tokens":2441.5,"median_padded_T":4096.0,"first_call_ms_median":19.99672901001759,"second_call_ms_median":18.970812496263534,"host_ms_median":4.222353512886912}},"all_first_call_ms_median":6.585145994904451,"all_second_call_ms_median":6.063770488253795},"processes":[{"pid":86441,"started":1791003222.662739,"load_model_seconds":0.09951466700294986,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.304162999993423,"t1024":0.004872457997407764,"t2048":0.0012187089887447655,"t4096":0.00125966701307334},"reset_check":{"run":["own_t01","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003224.6282291,"process_wall_seconds":2.430507957993541,"runs":36},{"pid":86508,"started":1791003225.1157892,"load_model_seconds":0.0051333750016056,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.09339399999589659,"t1024":0.0011110409977845848,"t2048":0.0011530829942785203,"t4096":0.004173958004685119},"reset_check":{"run":["img_03","native","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003226.5931678,"process_wall_seconds":1.961905583011685,"runs":36},{"pid":86575,"started":1791003227.075048,"load_model_seconds":0.005467875016620383,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08339737501228228,"t1024":0.0012489579967223108,"t2048":0.0011326660169288516,"t4096":0.004562749993056059},"reset_check":{"run":["semif_0b43ea8e24e74d621f1b","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003227.91094,"process_wall_seconds":1.318930959008867,"runs":36},{"pid":86617,"started":1791003228.378762,"load_model_seconds":0.005597957991994917,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08678620899445377,"t1024":0.001233959017554298,"t2048":0.0012225840182509273,"t4096":0.0011881660029757768},"reset_check":{"run":["semif_fda0ca94652c6d739ecf","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003229.213335,"process_wall_seconds":1.3134764159913175,"runs":36},{"pid":86673,"started":1791003229.6983428,"load_model_seconds":0.005259500001557171,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08341495800414123,"t1024":0.0010288329795002937,"t2048":0.0009603340004105121,"t4096":0.0009243749955203384},"reset_check":{"run":["semif_caf773b23934fa7362a1","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003230.530265,"process_wall_seconds":1.2999885830213316,"runs":36},{"pid":86688,"started":1791003231.015115,"load_model_seconds":0.00524570798734203,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08492350002052262,"t1024":0.0010825000063050538,"t2048":0.0009624579979572445,"t4096":0.0009584579966031015},"reset_check":{"run":["semif_0ebd299739302d8b057b","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003231.810724,"process_wall_seconds":1.2789651670027524,"runs":34}],"red_arm":null,"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max"},"script":{"gate_head.py":"1d66d14945d23d15b80866799a6287a08fcca41cf95ff56ea41e418a8c0b5e0b","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d"},"seconds":{"gpu_processes":9.61207191599533},"generated_at":"2026-10-03T04:53:52+00:00","runs":[{"id":"own_t01","arm":"text","variant":"base","tokens":436,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":1.2392080097924918,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.293381839990616e-08,"max_abs_dlogit":2.1457672119140625e-06},{"id":"own_t02","arm":"text","variant":"base","tokens":599,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,5],[5,7],[7,12],[12,14]],"host_ms":1.7531249905005097,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":6.693881005048752e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"own_t03","arm":"text","variant":"base","tokens":463,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":7,"layout":[[0,3],[3,5],[5,7]],"host_ms":0.6469170039054006,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.7939677238464355e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"own_t04","arm":"text","variant":"base","tokens":493,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,9],[9,11]],"host_ms":0.5953750223852694,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-09,"mean_abs_dp":3.344294699755582e-09,"max_abs_dlogit":5.960464477539062e-07},{"id":"own_t05","arm":"text","variant":"base","tokens":553,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,5],[5,7],[7,11],[11,13]],"host_ms":0.8241249888669699,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":6.071506784512446e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"own_t06","arm":"text","variant":"base","tokens":411,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.48808299470692873,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.0512531995773315e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"own_t07","arm":"text","variant":"base","tokens":438,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":12,"layout":[[0,6],[6,10],[10,12]],"host_ms":0.5924589931964874,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.979060471057892e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"own_t08","arm":"text","variant":"base","tokens":604,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,3],[3,5],[5,8],[8,18]],"host_ms":1.7070419853553176,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.7881393432617188e-07,"mean_abs_dp":1.9428423709339565e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"own_t09","arm":"text","variant":"base","tokens":582,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,4],[4,6],[6,9],[9,16]],"host_ms":0.8903330017346889,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.0372681319713593e-08,"max_abs_dlogit":2.9802322387695312e-06},{"id":"own_t10","arm":"text","variant":"base","tokens":396,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":8,"layout":[[0,2],[2,6],[6,8]],"host_ms":0.5139999848324805,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.725290298461914e-08,"mean_abs_dp":8.96397978067398e-09,"max_abs_dlogit":1.4901161193847656e-06},{"id":"own_t11","arm":"text","variant":"base","tokens":497,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,2],[2,4],[4,8],[8,11]],"host_ms":0.6835829990450293,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.836300568147139e-08,"max_abs_dlogit":1.3709068298339844e-06},{"id":"own_t12","arm":"text","variant":"base","tokens":967,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":29,"layout":[[0,2],[2,8],[8,10],[10,13],[13,18],[18,23],[23,25],[25,29]],"host_ms":1.7697920266073197,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.0858414214232874e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"own_t13","arm":"text","variant":"base","tokens":370,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":8,"layout":[[0,2],[2,5],[5,8]],"host_ms":0.43141699279658496,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.0826624929904938e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"own_t14","arm":"text","variant":"base","tokens":2389,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":15,"layout":[[0,5],[5,7],[7,11],[11,15]],"host_ms":5.295291019137949,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"own_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.0337680578231811e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"own_t15","arm":"text","variant":"base","tokens":2603,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":22,"layout":[[0,5],[5,7],[7,11],[11,13],[13,18],[18,22]],"host_ms":5.695250001735985,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"own_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.725290298461914e-08,"mean_abs_dp":4.1592019525441256e-09,"max_abs_dlogit":7.152557373046875e-07},{"id":"own_t16","arm":"text","variant":"base","tokens":2494,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":19,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17],[17,19]],"host_ms":2.2966249962337315,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"own_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":7.426072108118158e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"own_j01","arm":"text","variant":"base","tokens":476,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":0.5897499795537442,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":6.100162863731384e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"own_j02","arm":"text","variant":"base","tokens":475,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,2],[2,6],[6,8],[8,11]],"host_ms":0.5762499931734055,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.0573762330141934e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"own_j03","arm":"text","variant":"base","tokens":585,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,2],[2,7],[7,10],[10,14]],"host_ms":0.8895829960238189,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":7.650149720055716e-09,"max_abs_dlogit":1.430511474609375e-06},{"id":"own_j04","arm":"text","variant":"base","tokens":569,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,6],[6,11],[11,14]],"host_ms":0.8164169848896563,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":6.685565624918257e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"own_j05","arm":"text","variant":"base","tokens":539,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,11],[11,15]],"host_ms":0.7969999860506505,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.084610362847646e-08,"max_abs_dlogit":1.6689300537109375e-06},{"id":"own_j06","arm":"text","variant":"base","tokens":616,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,6],[6,8],[8,12],[12,14],[14,17]],"host_ms":0.8816249901428819,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.869493109338424e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"own_j07","arm":"text","variant":"base","tokens":595,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,5],[5,10],[10,14],[14,16]],"host_ms":0.8063749992288649,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.83936208486557e-08,"max_abs_dlogit":8.344650268554688e-07},{"id":"own_j08","arm":"text","variant":"base","tokens":473,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.7321670127566904,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.470348358154297e-08,"mean_abs_dp":1.862645149230957e-08,"max_abs_dlogit":5.066394805908203e-07},{"id":"own_j09","arm":"text","variant":"base","tokens":573,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":12,"layout":[[0,5],[5,7],[7,9],[9,12]],"host_ms":0.7625840080436319,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-07,"mean_abs_dp":3.3915663758913674e-08,"max_abs_dlogit":1.1920928955078125e-06},{"id":"own_j10","arm":"text","variant":"base","tokens":413,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,4],[4,9],[9,11]],"host_ms":0.5572499940171838,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":3.0140985142100944e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"own_j11","arm":"text","variant":"base","tokens":463,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.5497919919434935,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":3.078021109104156e-08,"max_abs_dlogit":1.1920928955078125e-06},{"id":"own_j12","arm":"text","variant":"base","tokens":562,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,2],[2,6],[6,10],[10,13]],"host_ms":0.7485839887522161,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.3497984959528996e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_01","arm":"g256","variant":"base","tokens":430,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,6],[6,8],[8,11]],"host_ms":0.5249590030871332,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.8203123049302536e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_01","arm":"g448","variant":"base","tokens":562,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,6],[6,8],[8,11]],"host_ms":0.6673340103588998,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.6246363466436214e-08,"max_abs_dlogit":1.7881393432617188e-06},{"id":"img_01","arm":"native","variant":"base","tokens":430,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,6],[6,8],[8,11]],"host_ms":0.5137080152053386,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.8203123049302536e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_02","arm":"g256","variant":"base","tokens":466,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,5],[5,7],[7,11]],"host_ms":0.5692499980796129,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-08,"mean_abs_dp":1.4393167062239214e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_02","arm":"g448","variant":"base","tokens":598,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,5],[5,7],[7,11]],"host_ms":0.6916669954080135,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.4976378137415104e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_02","arm":"native","variant":"base","tokens":702,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,5],[5,7],[7,11]],"host_ms":0.7371249957941473,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.785501154986295e-08,"max_abs_dlogit":2.0265579223632812e-06},{"id":"img_03","arm":"g256","variant":"base","tokens":495,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":15,"layout":[[0,10],[10,12],[12,15]],"host_ms":0.6242920062504709,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":5.587935447692871e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_03","arm":"g448","variant":"base","tokens":627,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,10],[10,12],[12,15]],"host_ms":0.7600410026498139,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":4.6876569588979084e-09,"max_abs_dlogit":8.344650268554688e-07},{"id":"img_03","arm":"native","variant":"base","tokens":501,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":15,"layout":[[0,10],[10,12],[12,15]],"host_ms":1.2886659824289382,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_04","arm":"g256","variant":"base","tokens":558,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,12],[12,15]],"host_ms":1.634665997698903,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.0978193283081055e-08,"mean_abs_dp":6.488213936487834e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_04","arm":"g448","variant":"base","tokens":690,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,12],[12,15]],"host_ms":1.7147080216091126,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.842877388000488e-08,"mean_abs_dp":5.805244048436483e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_04","arm":"native","variant":"base","tokens":1262,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,12],[12,15]],"host_ms":2.8702499985229224,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.5425106287002564e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_05","arm":"g256","variant":"base","tokens":464,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.5673329869750887,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.6679140654477207e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"img_05","arm":"g448","variant":"base","tokens":596,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.6904590118210763,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":8.974562991749157e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_05","arm":"native","variant":"base","tokens":464,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.582000007852912,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.6679140654477207e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"img_06","arm":"g256","variant":"base","tokens":465,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.5779589992016554,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.3655593395233153e-08,"max_abs_dlogit":6.556510925292969e-07},{"id":"img_06","arm":"g448","variant":"base","tokens":597,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.8239590097218752,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.451721906661987e-08,"max_abs_dlogit":1.3113021850585938e-06},{"id":"img_06","arm":"native","variant":"base","tokens":471,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.5784579843748361,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.2351741790771484e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_07","arm":"g256","variant":"base","tokens":426,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.5116250249557197,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.778826117515564e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"img_07","arm":"g448","variant":"base","tokens":558,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.6564169889315963,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.4808028936386109e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_07","arm":"native","variant":"base","tokens":662,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.676124996971339,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":6.426125764846802e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_08","arm":"g256","variant":"base","tokens":522,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,9],[9,11],[11,14]],"host_ms":0.739125011023134,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":3.5922442163739886e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_08","arm":"g448","variant":"base","tokens":654,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,9],[9,11],[11,14]],"host_ms":0.7384169730357826,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.470348358154297e-08,"mean_abs_dp":1.2772423880440848e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_08","arm":"native","variant":"base","tokens":1226,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,9],[9,11],[11,14]],"host_ms":1.2227910046931356,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.9602753264563423e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"img_09","arm":"g256","variant":"base","tokens":460,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.5512089992407709,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.1081756461750378e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_09","arm":"g448","variant":"base","tokens":592,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.6588749820366502,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.9303777001120827e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"img_09","arm":"native","variant":"base","tokens":696,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.7173749909270555,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.6425143588672985e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"img_10","arm":"g256","variant":"base","tokens":405,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.4820000031031668,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.1900232897864448e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_10","arm":"g448","variant":"base","tokens":537,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.6313750054687262,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":4.734223087628682e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"img_10","arm":"native","variant":"base","tokens":405,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.4753330140374601,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":4.242691728803846e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"photo_01","arm":"g256","variant":"base","tokens":556,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,6],[6,8],[8,13],[13,16]],"host_ms":0.8192500099539757,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":3.378954716026783e-08,"max_abs_dlogit":1.1324882507324219e-06},{"id":"photo_01","arm":"g448","variant":"base","tokens":688,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,6],[6,8],[8,13],[13,16]],"host_ms":0.9225840040016919,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.940696716308594e-08,"mean_abs_dp":2.5960616767406464e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"photo_01","arm":"native","variant":"base","tokens":1692,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":16,"layout":[[0,6],[6,8],[8,13],[13,16]],"host_ms":3.1494160066358745,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.7881393432617188e-07,"mean_abs_dp":2.7561327442526817e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"photo_02","arm":"g256","variant":"base","tokens":499,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,8],[8,11],[11,13]],"host_ms":0.6023750174790621,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.470348358154297e-08,"mean_abs_dp":5.229734457456148e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"photo_02","arm":"g448","variant":"base","tokens":631,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,8],[8,11],[11,13]],"host_ms":0.7292920199688524,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.210719347000122e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"photo_02","arm":"native","variant":"base","tokens":1035,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,8],[8,11],[11,13]],"host_ms":1.092082995455712,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.590412396651048e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"photo_03","arm":"g256","variant":"base","tokens":505,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,4],[4,8],[8,12]],"host_ms":0.6147919921204448,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-08,"mean_abs_dp":1.854884127775828e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"photo_03","arm":"g448","variant":"base","tokens":637,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,4],[4,8],[8,12]],"host_ms":0.754666980355978,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.6387472947438557e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"photo_03","arm":"native","variant":"base","tokens":1041,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,4],[4,8],[8,12]],"host_ms":1.094917010050267,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.7462298274040222e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"photo_04","arm":"g256","variant":"base","tokens":502,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":0.6317079823929816,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-09,"mean_abs_dp":2.149205941420335e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"photo_04","arm":"g448","variant":"base","tokens":634,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":0.7411249971482903,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.017290812272292e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"photo_04","arm":"native","variant":"base","tokens":1038,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":1.2519170122686774,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.0925130202220036e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_a3f18f3a63d45345942b","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.42875000508502126,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.0489096641540527e-08,"mean_abs_dp":9.313225746154785e-09,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_f40beba9088c8db8bbd6","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30029198387637734,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":1.3659397761027018e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_0b43ea8e24e74d621f1b","arm":"text","variant":"base","tokens":177,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.7881670026108623,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":8.692344029744467e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_942ec65dac0bfb9874f9","arm":"text","variant":"base","tokens":177,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.38495802436955273,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":6.084640820821126e-08,"max_abs_dlogit":1.1920928955078125e-06},{"id":"semif_e0c140e9222d1bce5647","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3997500170953572,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":5.960464477539062e-07},{"id":"semif_8e8c3804a3c15ebb31e3","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28491701232269406,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":1.862645149230957e-09,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_56c419b12cd37cbe790c","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28208299772813916,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":9.934107462565104e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_75b470586834df8c568d","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27070901705883443,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.470348358154297e-08,"mean_abs_dp":1.924733320871989e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_834f3b3e10cc61b33d25","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26745800278149545,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":6.20881716410319e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_76c7a79b1ff2363a60c5","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25891701807267964,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.687656958897909e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_36c71bb649b7d5eccb1f","arm":"text","variant":"base","tokens":174,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27120800223201513,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":2.1730860074361167e-09,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_eac5121a4a1bf391a08c","arm":"text","variant":"base","tokens":173,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2901249972637743,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":2.1730860074361167e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_44574c44e243ee1b2f52","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3736669896170497,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-09,"mean_abs_dp":3.725290298461914e-09,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_2608370d01783735c3ea","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.37312498898245394,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":5.339582761128744e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_ec94ef909f7918695ab3","arm":"text","variant":"base","tokens":180,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28849998489022255,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.7939677238464355e-08,"mean_abs_dp":1.30385160446167e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_03d919fd0e9b3a7cd1e6","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2752089931163937,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":8.692344029744467e-09,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_86a15618ba4cb90a307e","arm":"text","variant":"base","tokens":179,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27687501278705895,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":6.457169850667317e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_3dd9c3ac98f8ddf7bdd3","arm":"text","variant":"base","tokens":179,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27433299692347646,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":1.1175870895385742e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_36273dd4777876a26338","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27091699303127825,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.6763806343078613e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_07ee70cfe4b01330ce89","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.266042014118284,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":1.1175870895385742e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_51ceab3ce622c7c759f1","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3620420175138861,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-09,"mean_abs_dp":3.725290298461914e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_b8091be929b1e489fcc4","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2717910101637244,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.725290298461914e-08,"mean_abs_dp":2.2351741790771484e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_c89b86e3f55123bfa506","arm":"text","variant":"base","tokens":182,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2642920007929206,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0,"mean_abs_dp":0.0,"max_abs_dlogit":0.0},{"id":"semif_3b0958cabc339005cf66","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28974999440833926,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":3.725290298461914e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_f2b4ec4930322fe45118","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.42799999937415123,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-09,"mean_abs_dp":6.20881716410319e-10,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_61ac20b07a5189231e15","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3849170170724392,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":5.5258472760518394e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_4b3d794c004566610181","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27904199669137597,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.725290298461914e-09,"mean_abs_dp":2.483526865641276e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_b7c4e62d07986286fce0","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.32037499477155507,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.7318795522054035e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_9473603b44af63beab79","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3048340149689466,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":6.20881716410319e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_d41072dc35d8cd14c056","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30479198903776705,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.90496555964152e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_33d6e5da58f0fee2490d","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2907089947257191,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.6763806343078613e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_823bf736b92e9d8fa66d","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28895799187012017,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.705522537231445e-08,"mean_abs_dp":4.346172014872233e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_8a4c3de28be95c105ca1","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2869580057449639,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.9802322387695312e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_ea7ec66a5e8eac886548","arm":"text","variant":"base","tokens":192,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28316699899733067,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":3.60111395517985e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_25172cd6ddc49ab0ca70","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29045797418802977,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":1.0554989178975424e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_fb73a85d386cf59b0c51","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2964999875985086,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":8.568167686462402e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_a02fd582d2881cc462b2","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27650000993162394,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":1.862645149230957e-09,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_19f37d5a97034bd7d04f","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2749999985098839,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-09,"mean_abs_dp":3.104408582051595e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_fda0ca94652c6d739ecf","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.817291991552338,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.5456150372823078e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_d0e36c4cb10a29b3191c","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.362333026714623,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":7.698933283487956e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_a0a39ace7e5337d038a7","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4293750098440796,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.3151279290517173e-08,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_eeae283841ce75dd25bf","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2938329998869449,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-09,"mean_abs_dp":2.483526865641276e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_00bb4cc043f7d2b288d4","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27787499129772186,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.0489096641540527e-08,"mean_abs_dp":9.934107462565104e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_1a81ead69e93e5d0c86a","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28495799051597714,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-08,"mean_abs_dp":3.476937611897787e-08,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_4444dad25e438f128673","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28670899337157607,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-09,"mean_abs_dp":3.104408582051595e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_3226202ba1ada2570e9f","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3052090178243816,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":3.3527612686157227e-08,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_3f783eec5c90812479bc","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2894159988500178,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-09,"mean_abs_dp":6.20881716410319e-10,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_ffee861474a9665a093a","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2958750119432807,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.7881393432617188e-07,"mean_abs_dp":7.947285970052083e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_8c992e17e4fb54b044f5","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2752089931163937,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-09,"mean_abs_dp":1.241763432820638e-09,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_695dd61195ca025c4f4c","arm":"text","variant":"base","tokens":179,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2738329931162298,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":2.1730860074361167e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_e6155431ab7b0f8170d4","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.24454199592582881,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":4.035731156667073e-09,"max_abs_dlogit":5.960464477539062e-07},{"id":"semif_b49ecda75086d8c68d2a","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26950001483783126,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":6.953875223795573e-08,"max_abs_dlogit":1.6689300537109375e-06},{"id":"semif_587ced261c6092cd1325","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2520409761928022,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-10,"mean_abs_dp":3.104408582051595e-10,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_533d4423d311de82b27d","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29220798751339316,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":2.1730860074361167e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_2bf540d515a51755b25c","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3046249912586063,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":2.1730860074361167e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_fdb9fc214aea2710763a","arm":"text","variant":"base","tokens":168,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26954198256134987,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_f6d68126b43f8c9ed183","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2664579951670021,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.30385160446167e-08,"mean_abs_dp":4.346172014872233e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_44bae0c8a1615a096c43","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25087498943321407,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":3.4148494402567544e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_2175379f4c69e6207626","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2741249918472022,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":1.241763432820638e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_def9fe59d262357a673a","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26108400197699666,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1175870895385742e-08,"mean_abs_dp":3.725290298461914e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_712f6ffb763eef7c9a12","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27249997947365046,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":3.725290298461914e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_e1d610bd14e3d16c09b9","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26020800578407943,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.3969838619232178e-08,"mean_abs_dp":6.51925802230835e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_22ec57822db50ec121cf","arm":"text","variant":"base","tokens":171,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2607919741421938,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":3.4148494402567544e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_3e75f2b84e8e08e3d3f2","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2603330067358911,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.60770320892334e-08,"mean_abs_dp":1.3659397761027018e-08,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_ebd74aa05324a1556288","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2669160021468997,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-09,"mean_abs_dp":4.967053731282552e-09,"max_abs_dlogit":5.960464477539062e-07},{"id":"semif_d7e84e4245a29817da1f","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27154202689416707,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.190951585769653e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_f41ae66b3c09630c7a42","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2969999914057553,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.3969838619232178e-08,"mean_abs_dp":9.002784887949625e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_82c2cd97c160ad27951b","arm":"text","variant":"base","tokens":171,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2608749782666564,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.940696716308594e-08,"mean_abs_dp":2.9802322387695312e-08,"max_abs_dlogit":1.1920928955078125e-06},{"id":"semif_7ada97b6f99f4788e5c7","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26666701887734234,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-09,"mean_abs_dp":3.104408582051595e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_4d7a003342db5b0a17e3","arm":"text","variant":"base","tokens":187,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29629102209582925,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-09,"mean_abs_dp":2.483526865641276e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_19d3d8d0bf773e88be7b","arm":"text","variant":"base","tokens":172,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2580000145826489,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.210719347000122e-08,"mean_abs_dp":5.89837630589803e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_d6de731cca52359b1c87","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27220899937674403,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.21540641784668e-08,"mean_abs_dp":1.9868214925130207e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_b45c63fb92729af27f05","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2999170101247728,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":1.241763432820638e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_46b7029b9a704138b77a","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.34729199251160026,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-09,"mean_abs_dp":6.20881716410319e-10,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_caf773b23934fa7362a1","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.7365829951595515,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.421438694000244e-08,"mean_abs_dp":1.1175870895385742e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_f46f392ef9e9e9df564b","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3406250034458935,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.7807892163594566e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_005dcf6d2c0f21d64796","arm":"text","variant":"base","tokens":209,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.45841699466109276,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1175870895385742e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_edb708bd0fe5fd67d6ee","arm":"text","variant":"base","tokens":210,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4482500080484897,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1175870895385742e-08,"mean_abs_dp":4.967053731282552e-09,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_97a35499bd8717f3e786","arm":"text","variant":"base","tokens":203,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2929579932242632,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.421438694000244e-08,"mean_abs_dp":8.692344029744467e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_31eb4533d2c89959ec46","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28733297949656844,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":5.712111790974935e-08,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_0aa24a6701ac9c56c4a0","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2957090036943555,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.594524701436361e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_796d5c0da6eaadae6a3f","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27179199969395995,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":2.1730860074361167e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_599b3092b6fd18312d23","arm":"text","variant":"base","tokens":187,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2522919967304915,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.7807892163594566e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_960a4bcd27363e6db4b9","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2612080133985728,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.7881393432617188e-07,"mean_abs_dp":9.189049402872722e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_c6a411e7f2bbdf377173","arm":"text","variant":"base","tokens":207,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2930000191554427,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.60770320892334e-08,"mean_abs_dp":8.692344029744467e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_e332aa927d0c2ed044aa","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2823329996317625,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":5.2774945894877114e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"semif_a0128e551d095f3449f8","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2970420173369348,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":3.725290298461914e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_46d6a0731bd288c370f0","arm":"text","variant":"base","tokens":182,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2920409897342324,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.60770320892334e-08,"mean_abs_dp":1.6142924626668293e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_21ca7a8b970e36b8e20f","arm":"text","variant":"base","tokens":214,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.40620798245072365,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":3.9736429850260414e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_4aa83af95bb69e90bc2d","arm":"text","variant":"base","tokens":212,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30691601568832994,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.341104507446289e-07,"mean_abs_dp":8.940696716308594e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_cdfa9eed51fafcf09b72","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2839170047082007,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":3.725290298461914e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_d51b05bfb9d6d7f09331","arm":"text","variant":"base","tokens":203,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3009169886354357,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.3527612686157227e-08,"mean_abs_dp":2.1109978357950847e-08,"max_abs_dlogit":5.960464477539062e-07},{"id":"semif_fdd902cc6e65fad7cb4f","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27383302222006023,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_42384af9ae56beb37ff5","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2882499829865992,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.6763806343078613e-08,"mean_abs_dp":7.761021455128988e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_cdf7f428857e1b841cb3","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26591698406264186,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":2.1730860074361167e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_774f7f9245f83c796d44","arm":"text","variant":"base","tokens":180,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2934579970315099,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.7939677238464355e-08,"mean_abs_dp":1.4280279477437338e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_92ec2ffa3dd46ada1335","arm":"text","variant":"base","tokens":204,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30112499371171,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":5.587935447692871e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_6149a17bc154f9c5b4a0","arm":"text","variant":"base","tokens":200,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.276917009614408,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":8.692344029744467e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_be866e1dbec5d42d9423","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29554200591519475,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0,"mean_abs_dp":0.0,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_97d2ed2591fc7638806c","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2787080011330545,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":3.2285849253336586e-08,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_4533dd91267b4d815690","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2792919985949993,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1175870895385742e-08,"mean_abs_dp":5.587935447692871e-09,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_d880e65fef5e2993fdd0","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2852079924196005,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":3.476937611897787e-08,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_90494c76078ffc3c717a","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2745420206338167,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":6.20881716410319e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_9ac5cc82c4126c91174a","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29845800600014627,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":9.934107462565104e-09,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_386e10b1c1ce906473a4","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2983330050483346,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-09,"mean_abs_dp":3.4148494402567544e-09,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_4da6aedc1139224eeff2","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26408297708258033,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0,"mean_abs_dp":0.0,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_a22dab8ba073ff6cdb8d","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2623339823912829,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.51925802230835e-09,"mean_abs_dp":4.346172014872233e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_42c54fa2ad45743e24d8","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2546669857110828,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":7.450580596923828e-09,"max_abs_dlogit":5.960464477539062e-07},{"id":"semif_f3d13cf184f916677def","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2833339967764914,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":5.339582761128744e-08,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_1105577da4c8dad4609d","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.277625018497929,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":5.463759104410807e-08,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_0ebd299739302d8b057b","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.7621669792570174,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.9802322387695312e-08,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_eafc22c8c40df3932a8e","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.31108397524803877,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":1.241763432820638e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_0aa1447680e4f25862bb","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.468041020212695,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":5.21540641784668e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_980e76145c925747220c","arm":"text","variant":"base","tokens":209,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.43074999121017754,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":4.2219956715901695e-08,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_4aee37a6a155747766aa","arm":"text","variant":"base","tokens":206,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27916699764318764,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0,"mean_abs_dp":0.0,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_6db17cc22bd656d78558","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2820419904310256,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.594524701436361e-08,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_ef3e0f8e657bf59f8a29","arm":"text","variant":"base","tokens":200,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2880830143112689,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":8.940696716308594e-08,"max_abs_dlogit":1.6689300537109375e-06},{"id":"semif_f756e1cb04e11766341e","arm":"text","variant":"base","tokens":206,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2719589974731207,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":5.463759104410807e-08,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_d167e51bb1f499099645","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2668329980224371,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.7881393432617188e-07,"mean_abs_dp":9.313225746154785e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"semif_674ca4e9346216255466","arm":"text","variant":"base","tokens":176,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25658399681560695,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":2.483526865641276e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_1df851a202994a07f123","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3442919987719506,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":6.20881716410319e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_b5e6788d95b8395d2888","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2717080060392618,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.381903171539307e-09,"mean_abs_dp":3.4148494402567544e-09,"max_abs_dlogit":5.960464477539063e-08},{"id":"semif_32f36cd0fbbfb3604586","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2597499988041818,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":8.692344029744467e-09,"max_abs_dlogit":3.5762786865234375e-07},{"id":"semif_843185706adff0c60f8b","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28720899717882276,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.9868214925130207e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_b531dd603bf080dab820","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29033300234004855,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-08,"mean_abs_dp":6.829698880513509e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_be2dd67bc733c1213bef","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3137500025331974,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.6566128730773926e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_b8e2750307df9761e716","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3032500098925084,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-09,"mean_abs_dp":4.346172014872233e-09,"max_abs_dlogit":2.384185791015625e-07},{"id":"semif_5d31b4bf2d0fc193f53b","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2834589977283031,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":9.313225746154785e-09,"max_abs_dlogit":5.960464477539062e-07},{"id":"semif_a9230edfc6ec9e524af0","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27150000096298754,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.313225746154785e-10,"mean_abs_dp":3.104408582051595e-10,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_c100e84585149a04be49","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2690830151550472,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.30385160446167e-08,"mean_abs_dp":6.51925802230835e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_579beed184ffc1104aa9","arm":"text","variant":"base","tokens":182,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26962501578964293,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":4.2219956715901695e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_c4e9bd463b6fde5be743","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26504098786972463,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.9802322387695312e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_b863998d4be2721a5dfd","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2750830026343465,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0,"mean_abs_dp":0.0,"max_abs_dlogit":0.0},{"id":"semif_37cebc3792e98049439f","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26783300563693047,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.587935447692871e-09,"mean_abs_dp":3.104408582051595e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_64d63f86e328aff1481c","arm":"text","variant":"base","tokens":210,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3862089943140745,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.862645149230957e-08,"mean_abs_dp":9.313225746154785e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_6d64a8459f64eac7f4cc","arm":"text","variant":"base","tokens":207,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2758750051725656,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1175870895385742e-08,"mean_abs_dp":5.587935447692871e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_2708193212d8a4a523c7","arm":"text","variant":"base","tokens":207,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2877080114558339,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.7881393432617188e-07,"mean_abs_dp":1.043081283569336e-07,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_5a37d1dfcad20810a76a","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30633402639068663,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":6.022552649180095e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_acacd8b46acd25cb954b","arm":"text","variant":"base","tokens":206,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27933300589211285,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1175870895385742e-08,"mean_abs_dp":6.20881716410319e-09,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_4089c199b8b796ec3ed8","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2994170063175261,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":6.953875223795573e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_02f2b95a5c5a4777a45c","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2646659850142896,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.450580596923828e-09,"mean_abs_dp":2.483526865641276e-09,"max_abs_dlogit":1.1920928955078125e-07},{"id":"semif_ebea4681e3dcc7d92ca2","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2932499919552356,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.470348358154297e-08,"mean_abs_dp":2.1109978357950847e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"semif_b8c5b9b285cbdddc9f8d","arm":"text","variant":"base","tokens":187,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26912501198239625,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.30385160446167e-08,"mean_abs_dp":4.346172014872233e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"semif_7ed75d854d99b3a81916","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2606250054668635,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1175870895385742e-08,"mean_abs_dp":6.20881716410319e-09,"max_abs_dlogit":4.76837158203125e-07}]},"h1_heldout":{"schema":"clef-flash-head-h1/1","gate":"head graph alone on the oracle's fp32 hidden (lexical from the fp16 table)","bar":{"max_abs_dp":0.001,"argmax":"every question"},"checks":{"all_runs":true,"argmax_every_question":true,"max_abs_dp":true,"finite_and_padding_zero":true,"rerun_bit_equal":true},"result":"PASS","summary":{"runs":40,"questions":188,"argmax_equal":188,"questions_non_near_tie":186,"argmax_equal_non_near_tie":186,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":1.7881393432617188e-07,"mean_of_run_mean_abs_dp":1.7395698404004317e-08,"max_abs_dlogit":3.874301910400391e-06,"worst_run":"ho_t06/text"},"by_source_arm":{"heldout_image/g256":{"runs":8,"questions":31,"argmax_equal":31,"questions_non_near_tie":31,"argmax_equal_non_near_tie":31,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.2620147714425633e-08,"max_abs_dlogit":1.430511474609375e-06,"worst_run":"ho_i01/g256"},"heldout_image/g448":{"runs":8,"questions":31,"argmax_equal":31,"questions_non_near_tie":31,"argmax_equal_non_near_tie":31,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.3773902260054398e-08,"max_abs_dlogit":3.874301910400391e-06,"worst_run":"ho_i01/g448"},"heldout_json/text":{"runs":8,"questions":36,"argmax_equal":36,"questions_non_near_tie":35,"argmax_equal_non_near_tie":35,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":2.1457152415033637e-08,"max_abs_dlogit":9.5367431640625e-07,"worst_run":"ho_j01/text"},"heldout_photo/g256":{"runs":2,"questions":10,"argmax_equal":10,"questions_non_near_tie":10,"argmax_equal_non_near_tie":10,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.7645823604920333e-08,"max_abs_dlogit":1.5497207641601562e-06,"worst_run":"ho_p01/g256"},"heldout_photo/g448":{"runs":2,"questions":10,"argmax_equal":10,"questions_non_near_tie":10,"argmax_equal_non_near_tie":10,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.1920928955078125e-07,"mean_of_run_mean_abs_dp":1.0097088913122813e-08,"max_abs_dlogit":9.5367431640625e-07,"worst_run":"ho_p01/g448"},"heldout_text/text":{"runs":10,"questions":56,"argmax_equal":56,"questions_non_near_tie":55,"argmax_equal_non_near_tie":55,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":1.7881393432617188e-07,"mean_of_run_mean_abs_dp":2.2697463418162498e-08,"max_abs_dlogit":2.6226043701171875e-06,"worst_run":"ho_t06/text"},"heldout_text_long/text":{"runs":2,"questions":14,"argmax_equal":14,"questions_non_near_tie":14,"argmax_equal_non_near_tie":14,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":1.4901161193847656e-07,"mean_of_run_mean_abs_dp":1.527892891317606e-08,"max_abs_dlogit":1.9073486328125e-06,"worst_run":"ho_l01/text"}},"head_bundle":{"dir":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","metadata_sha256":"631219aafcdb56ef1f008e5476f0e7a282933abdc51b4835132676d97124ab20","aimodelc":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_metadata":{"graph":"conversion/clef_flash/clef_head.py ClefHeadGraph: the author's JointSchemaHead for one record, tensor inputs only","shape":"bucket","functions":["t512","t1024","t2048","t4096"],"buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"dims":null,"host_pad":"bucket","host_pad_rule":"T to the smallest bucket >= T (512, 1024, 2048, 4096), Q to 16, O to 128; padding rows / columns zero, their valid flags 0","inputs":{"hidden":"[T, 4096] f32","key_valid":"[T] f32","q_avg":"[Q, T] f32","o_avg":"[O, T] f32","g_avg":"[1, T] f32","lexical":"[O, 4096] f32","member":"[Q, O] f32","type_ids":"[Q] i32","q_valid":"[Q] f32","o_valid":"[O] f32"},"input_order":["hidden","key_valid","q_avg","o_avg","g_avg","lexical","member","type_ids","q_valid","o_valid"],"outputs":{"logits":"[O] f32; padding options 0"},"weights":"fp16w32","softmax":"host: per question, fp32, over that question's options","lexical":"host: lm_head_fp16.bin rows of each option span's ids, averaged (clef_head.lexical_rows)","source":{"hf_id":"Cloudflare/clef-flash","revision":"17f0b0ad64efb65d273590632833508766b2aae6","joint_schema_model_sha256":"0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3","joint_head_safetensors_sha256":"19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0","joint_head_config_sha256":"77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0"}}},"descriptors":{"t512":{"function":"t512","inputs":{"hidden":[[512,4096],"float32"],"key_valid":[[512],"float32"],"q_avg":[[16,512],"float32"],"o_avg":[[128,512],"float32"],"g_avg":[[1,512],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t1024":{"function":"t1024","inputs":{"hidden":[[1024,4096],"float32"],"key_valid":[[1024],"float32"],"q_avg":[[16,1024],"float32"],"o_avg":[[128,1024],"float32"],"g_avg":[[1,1024],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t2048":{"function":"t2048","inputs":{"hidden":[[2048,4096],"float32"],"key_valid":[[2048],"float32"],"q_avg":[[16,2048],"float32"],"o_avg":[[128,2048],"float32"],"g_avg":[[1,2048],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t4096":{"function":"t4096","inputs":{"hidden":[[4096,4096],"float32"],"key_valid":[[4096],"float32"],"q_avg":[[16,4096],"float32"],"o_avg":[[128,4096],"float32"],"g_avg":[[1,4096],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}}},"descriptors_same_every_process":true,"oracle":{"path":"<lane>/oracle/heldout/records_oracle.json","sha256":"85e4dda56890dff47a163bdd74c85291f5d3be0944403dd1f59ba2f5b9faaa3f","runs":40},"table":{"path":"<lane>/exports/host/lm_head_fp16.bin","json":"<lane>/exports/host/lm_head_fp16.json"},"near_ties":[{"id":"ho_t07","arm":"text","question_id":"questions_asked","oracle_top2_margin":0.011684805154800415,"argmax_equal":true,"max_abs_dp":8.940696716308594e-08,"probs":[0.00985698401927948,0.4101867973804474,0.39850205183029175,0.18145424127578735],"probs_oracle":[0.009856987744569778,0.410186767578125,0.3985019624233246,0.18145422637462616]},{"id":"ho_j01","arm":"text","question_id":"late_start","oracle_top2_margin":0.0010575354099273682,"argmax_equal":true,"max_abs_dp":1.1920928955078125e-07,"probs":[0.500528872013092,0.49947112798690796],"probs_oracle":[0.5005287528038025,0.4994712173938751]}],"timing":{"contended":true,"by_T":{"300<T<=700":{"runs":16,"median_tokens":595.5,"median_padded_T":1024.0,"first_call_ms_median":8.478249990730546,"second_call_ms_median":7.870563000324182,"host_ms_median":0.812603990198113},"700<T<=1500":{"runs":22,"median_tokens":767.5,"median_padded_T":1024.0,"first_call_ms_median":8.516854009940289,"second_call_ms_median":7.904646001406945,"host_ms_median":1.0026250092778355},"T>1500":{"runs":2,"median_tokens":2790.5,"median_padded_T":4096.0,"first_call_ms_median":20.600812495104037,"second_call_ms_median":19.4472709990805,"host_ms_median":4.420583485625684}},"all_first_call_ms_median":8.516854009940289,"all_second_call_ms_median":7.9004169965628535},"processes":[{"pid":19294,"started":1791004729.67586,"load_model_seconds":0.005526875000214204,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08618354200734757,"t1024":0.0010989999864250422,"t2048":0.0011287079832982272,"t4096":0.0009919579897541553},"reset_check":{"run":["ho_t01","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791004731.575739,"process_wall_seconds":2.372184291976737,"runs":40}],"red_arm":null,"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max"},"script":{"gate_head.py":"1d66d14945d23d15b80866799a6287a08fcca41cf95ff56ea41e418a8c0b5e0b","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d"},"seconds":{"gpu_processes":2.3748991249885876},"generated_at":"2026-10-03T05:18:52+00:00","runs":[{"id":"ho_t01","arm":"text","variant":"base","tokens":708,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17]],"host_ms":2.0823329978156835,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.2516092833350686e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_t02","arm":"text","variant":"base","tokens":746,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,2],[2,7],[7,9],[9,13],[13,16]],"host_ms":2.0918330119457096,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.7462298274040222e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_t03","arm":"text","variant":"base","tokens":842,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":20,"layout":[[0,2],[2,6],[6,8],[8,12],[12,16],[16,20]],"host_ms":1.55387501581572,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.362648956477642e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_t04","arm":"text","variant":"base","tokens":658,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,5],[5,7],[7,9],[9,16]],"host_ms":0.8899170206859708,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":8.381903171539307e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_t05","arm":"text","variant":"base","tokens":727,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,6],[6,8],[8,10],[10,14]],"host_ms":0.9819169936235994,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.8759497574397495e-08,"max_abs_dlogit":1.1324882507324219e-06},{"id":"ho_t06","arm":"text","variant":"base","tokens":721,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,7],[7,11],[11,13],[13,17]],"host_ms":1.083332987036556,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.7881393432617188e-07,"mean_abs_dp":5.067490479525398e-08,"max_abs_dlogit":2.6226043701171875e-06},{"id":"ho_t07","arm":"text","variant":"base","tokens":1086,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":29,"layout":[[0,5],[5,9],[9,13],[13,15],[15,17],[17,21],[21,25],[25,29]],"host_ms":3.65279198740609,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.1564934788079097e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_t08","arm":"text","variant":"base","tokens":864,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":20,"layout":[[0,5],[5,9],[9,13],[13,15],[15,17],[17,20]],"host_ms":1.119499997003004,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.4877878129482269e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"ho_t09","arm":"text","variant":"base","tokens":867,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,3],[3,5],[5,7],[7,11],[11,15],[15,18]],"host_ms":1.0653329954948276,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.153683453798294e-08,"max_abs_dlogit":8.344650268554688e-07},{"id":"ho_t10","arm":"text","variant":"base","tokens":800,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":21,"layout":[[0,5],[5,7],[7,9],[9,13],[13,17],[17,21]],"host_ms":1.0233339853584766,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-07,"mean_abs_dp":2.757380051272256e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"ho_j01","arm":"text","variant":"base","tokens":769,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,8],[8,10],[10,12],[12,15]],"host_ms":0.9137080051004887,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.7567148208618165e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_j02","arm":"text","variant":"base","tokens":335,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":6,"layout":[[0,6]],"host_ms":0.6030000222381204,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.3283064365386963e-08,"max_abs_dlogit":2.384185791015625e-07},{"id":"ho_j03","arm":"text","variant":"base","tokens":775,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":19,"layout":[[0,2],[2,8],[8,12],[12,14],[14,19]],"host_ms":1.0210000036749989,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.8430383581864207e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_j04","arm":"text","variant":"base","tokens":610,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,6],[6,10],[10,12]],"host_ms":0.8209999941755086,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.1098260680834452e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_j05","arm":"text","variant":"base","tokens":775,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,7],[7,9],[9,13],[13,17]],"host_ms":0.9647079859860241,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.1228676333146936e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_j06","arm":"text","variant":"base","tokens":733,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,2],[2,5],[5,7],[7,11],[11,13],[13,18]],"host_ms":0.9631249995436519,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.6853134234746296e-08,"max_abs_dlogit":7.152557373046875e-07},{"id":"ho_j07","arm":"text","variant":"base","tokens":826,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,5],[5,10],[10,14],[14,16],[16,18]],"host_ms":1.0680000123102218,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.4842953532934189e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_j08","arm":"text","variant":"base","tokens":798,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,2],[2,7],[7,11],[11,14],[14,18]],"host_ms":1.0520420037209988,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.043081283569336e-07,"mean_abs_dp":2.83535983827379e-08,"max_abs_dlogit":8.940696716308594e-07},{"id":"ho_l01","arm":"text","variant":"base","tokens":2850,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":25,"layout":[[0,5],[5,9],[9,11],[11,15],[15,17],[17,21],[21,25]],"host_ms":6.407749984646216,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"heldout_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4901161193847656e-07,"mean_abs_dp":1.4230608940124511e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_l02","arm":"text","variant":"base","tokens":2731,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":24,"layout":[[0,5],[5,9],[9,11],[11,15],[15,19],[19,21],[21,24]],"host_ms":2.4334169866051525,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"heldout_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.6327248886227608e-08,"max_abs_dlogit":1.9073486328125e-06},{"id":"ho_i01","arm":"g256","variant":"base","tokens":529,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":0.7915000023785979,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.9199573076688328e-08,"max_abs_dlogit":3.5762786865234375e-07},{"id":"ho_i01","arm":"g448","variant":"base","tokens":661,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":0.749250000808388,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.8763539516008816e-08,"max_abs_dlogit":4.76837158203125e-07},{"id":"ho_i02","arm":"g256","variant":"base","tokens":570,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,8],[8,12],[12,14]],"host_ms":0.8129999914672226,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.1508486100605556e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_i02","arm":"g448","variant":"base","tokens":702,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,8],[8,12],[12,14]],"host_ms":0.8213330002035946,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.2838946921484812e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_i03","arm":"g256","variant":"base","tokens":541,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,10],[10,14],[14,16]],"host_ms":0.8122079889290035,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.30385160446167e-08,"mean_abs_dp":3.7834979593753815e-09,"max_abs_dlogit":1.430511474609375e-06},{"id":"ho_i03","arm":"g448","variant":"base","tokens":673,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,10],[10,14],[14,16]],"host_ms":0.8449589950032532,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2351741790771484e-08,"mean_abs_dp":4.452886059880257e-09,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_i04","arm":"g256","variant":"base","tokens":437,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":0.5459999956656247,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.6577541828155518e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_i04","arm":"g448","variant":"base","tokens":569,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":0.6963750056456774,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":2.0209699869155883e-08,"max_abs_dlogit":1.7285346984863281e-06},{"id":"ho_i05","arm":"g256","variant":"base","tokens":701,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":21,"layout":[[0,7],[7,11],[11,15],[15,17],[17,21]],"host_ms":0.9480000007897615,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.16082706621715e-08,"max_abs_dlogit":1.430511474609375e-06},{"id":"ho_i05","arm":"g448","variant":"base","tokens":833,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":21,"layout":[[0,7],[7,11],[11,15],[15,17],[17,21]],"host_ms":0.9842500148806721,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":5.344017630531674e-09,"max_abs_dlogit":1.9073486328125e-06},{"id":"ho_i06","arm":"g256","variant":"base","tokens":628,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":19,"layout":[[0,10],[10,15],[15,17],[17,19]],"host_ms":0.8508750179316849,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":6.102613712611951e-09,"max_abs_dlogit":1.430511474609375e-06},{"id":"ho_i06","arm":"g448","variant":"base","tokens":760,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":19,"layout":[[0,10],[10,15],[15,17],[17,19]],"host_ms":0.9309169836342335,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":8.957852658472563e-09,"max_abs_dlogit":7.152557373046875e-07},{"id":"ho_i07","arm":"g256","variant":"base","tokens":535,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,7],[7,11],[11,13]],"host_ms":0.746000005165115,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.2787775351450993e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_i07","arm":"g448","variant":"base","tokens":667,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,7],[7,11],[11,13]],"host_ms":0.7597919902764261,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.83936208486557e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_i08","arm":"g256","variant":"base","tokens":581,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,10],[10,12],[12,17]],"host_ms":0.8264589996542782,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.9393423024345847e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_i08","arm":"g448","variant":"base","tokens":713,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,10],[10,12],[12,17]],"host_ms":0.8420410158578306,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.1230654576245476e-08,"max_abs_dlogit":3.874301910400391e-06},{"id":"ho_p01","arm":"g256","variant":"base","tokens":634,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,2],[2,6],[6,8],[8,12],[12,15]],"host_ms":0.8623749890830368,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":36,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":2.3376196622848512e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_p01","arm":"g448","variant":"base","tokens":766,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,2],[2,6],[6,8],[8,12],[12,15]],"host_ms":0.8891249890439212,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":37,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1920928955078125e-07,"mean_abs_dp":1.4839073022206625e-08,"max_abs_dlogit":9.5367431640625e-07},{"id":"ho_p02","arm":"g256","variant":"base","tokens":620,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17]],"host_ms":0.8506670128554106,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":38,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.960464477539063e-08,"mean_abs_dp":1.1915450586992152e-08,"max_abs_dlogit":1.5497207641601562e-06},{"id":"ho_p02","arm":"g448","variant":"base","tokens":752,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17]],"host_ms":0.88825001148507,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":39,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.9802322387695312e-08,"mean_abs_dp":5.3551048040390015e-09,"max_abs_dlogit":9.5367431640625e-07}]},"h2_fixture":{"schema":"clef-flash-head-h2/1","gate":"end to end in Python: the decoder bundle's fp16 hidden (readout shards, not re-run) -> host span matrices + fp16-table lexical -> head graph -> per-question fp32 softmax","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02},"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite_and_padding_zero":true,"rerun_bit_equal":true},"result":"PASS","summary":{"runs":213,"questions":401,"argmax_equal":401,"questions_non_near_tie":397,"argmax_equal_non_near_tie":397,"near_tie_questions":4,"argmax_equal_near_tie":4,"max_abs_dp":0.011243820190429688,"mean_of_run_mean_abs_dp":0.0005416699218630339,"max_abs_dlogit":0.10230028629302979,"worst_run":"img_07/g256"},"by_source_arm":{"cc0_photo/g256":{"runs":4,"questions":16,"argmax_equal":16,"questions_non_near_tie":15,"argmax_equal_non_near_tie":15,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.0037543773651123047,"mean_of_run_mean_abs_dp":0.0002360411483842211,"max_abs_dlogit":0.01890707015991211,"worst_run":"photo_01/g256"},"cc0_photo/g448":{"runs":4,"questions":16,"argmax_equal":16,"questions_non_near_tie":16,"argmax_equal_non_near_tie":16,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.005205422639846802,"mean_of_run_mean_abs_dp":0.00027710009136428247,"max_abs_dlogit":0.016104936599731445,"worst_run":"photo_01/g448"},"cc0_photo/native":{"runs":3,"questions":12,"argmax_equal":12,"questions_non_near_tie":12,"argmax_equal_non_near_tie":12,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.00018852949142456055,"mean_of_run_mean_abs_dp":5.4958495351239144e-05,"max_abs_dlogit":0.011218070983886719,"worst_run":"photo_03/native"},"own_image/g256":{"runs":10,"questions":34,"argmax_equal":34,"questions_non_near_tie":33,"argmax_equal_non_near_tie":33,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.011243820190429688,"mean_of_run_mean_abs_dp":0.0006499055615917901,"max_abs_dlogit":0.05287814140319824,"worst_run":"img_07/g256"},"own_image/g448":{"runs":10,"questions":34,"argmax_equal":34,"questions_non_near_tie":33,"argmax_equal_non_near_tie":33,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.002756863832473755,"mean_of_run_mean_abs_dp":0.00024629054244333534,"max_abs_dlogit":0.024670958518981934,"worst_run":"img_06/g448"},"own_image/native":{"runs":10,"questions":34,"argmax_equal":34,"questions_non_near_tie":33,"argmax_equal_non_near_tie":33,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.0013368427753448486,"mean_of_run_mean_abs_dp":0.00019349524795654274,"max_abs_dlogit":0.01906609535217285,"worst_run":"img_01/native"},"own_json/text":{"runs":12,"questions":45,"argmax_equal":45,"questions_non_near_tie":45,"argmax_equal_non_near_tie":45,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.005557417869567871,"mean_of_run_mean_abs_dp":0.0004030869961145602,"max_abs_dlogit":0.03170323371887207,"worst_run":"own_j04/text"},"own_text/text":{"runs":13,"questions":50,"argmax_equal":50,"questions_non_near_tie":50,"argmax_equal_non_near_tie":50,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.00822708010673523,"mean_of_run_mean_abs_dp":0.0005152467692749358,"max_abs_dlogit":0.05237102508544922,"worst_run":"own_t03/text"},"own_text_long/text":{"runs":3,"questions":16,"argmax_equal":16,"questions_non_near_tie":16,"argmax_equal_non_near_tie":16,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.001191943883895874,"mean_of_run_mean_abs_dp":0.00013957957322236421,"max_abs_dlogit":0.034476637840270996,"worst_run":"own_t14/text"},"semif_authored144/text":{"runs":144,"questions":144,"argmax_equal":144,"questions_non_near_tie":144,"argmax_equal_non_near_tie":144,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.010642945766448975,"mean_of_run_mean_abs_dp":0.0006271343660989294,"max_abs_dlogit":0.10230028629302979,"worst_run":"semif_f756e1cb04e11766341e/text"}},"head_bundle":{"dir":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","metadata_sha256":"631219aafcdb56ef1f008e5476f0e7a282933abdc51b4835132676d97124ab20","aimodelc":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_metadata":{"graph":"conversion/clef_flash/clef_head.py ClefHeadGraph: the author's JointSchemaHead for one record, tensor inputs only","shape":"bucket","functions":["t512","t1024","t2048","t4096"],"buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"dims":null,"host_pad":"bucket","host_pad_rule":"T to the smallest bucket >= T (512, 1024, 2048, 4096), Q to 16, O to 128; padding rows / columns zero, their valid flags 0","inputs":{"hidden":"[T, 4096] f32","key_valid":"[T] f32","q_avg":"[Q, T] f32","o_avg":"[O, T] f32","g_avg":"[1, T] f32","lexical":"[O, 4096] f32","member":"[Q, O] f32","type_ids":"[Q] i32","q_valid":"[Q] f32","o_valid":"[O] f32"},"input_order":["hidden","key_valid","q_avg","o_avg","g_avg","lexical","member","type_ids","q_valid","o_valid"],"outputs":{"logits":"[O] f32; padding options 0"},"weights":"fp16w32","softmax":"host: per question, fp32, over that question's options","lexical":"host: lm_head_fp16.bin rows of each option span's ids, averaged (clef_head.lexical_rows)","source":{"hf_id":"Cloudflare/clef-flash","revision":"17f0b0ad64efb65d273590632833508766b2aae6","joint_schema_model_sha256":"0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3","joint_head_safetensors_sha256":"19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0","joint_head_config_sha256":"77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0"}}},"descriptors":{"t512":{"function":"t512","inputs":{"hidden":[[512,4096],"float32"],"key_valid":[[512],"float32"],"q_avg":[[16,512],"float32"],"o_avg":[[128,512],"float32"],"g_avg":[[1,512],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t1024":{"function":"t1024","inputs":{"hidden":[[1024,4096],"float32"],"key_valid":[[1024],"float32"],"q_avg":[[16,1024],"float32"],"o_avg":[[128,1024],"float32"],"g_avg":[[1,1024],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t2048":{"function":"t2048","inputs":{"hidden":[[2048,4096],"float32"],"key_valid":[[2048],"float32"],"q_avg":[[16,2048],"float32"],"o_avg":[[128,2048],"float32"],"g_avg":[[1,2048],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t4096":{"function":"t4096","inputs":{"hidden":[[4096,4096],"float32"],"key_valid":[[4096],"float32"],"q_avg":[[16,4096],"float32"],"o_avg":[[128,4096],"float32"],"g_avg":[[1,4096],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}}},"descriptors_same_every_process":true,"oracle":{"path":"<lane>/oracle/records_oracle.json","sha256":"d8adaa746a5d575e654212ca78f5634af334aaa62eb53ef150d5d5925b1fa7c0","runs":214},"table":{"path":"<lane>/exports/host/lm_head_fp16.bin","json":"<lane>/exports/host/lm_head_fp16.json"},"near_ties":[{"id":"img_01","arm":"g256","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"max_abs_dp":0.0006841421127319336,"probs":[0.4912445545196533,0.487699419260025,0.02105601318180561],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_01","arm":"native","question_id":"clutter","oracle_top2_margin":0.002184569835662842,"argmax_equal":true,"max_abs_dp":0.0006841421127319336,"probs":[0.4912445545196533,0.487699419260025,0.02105601318180561],"probs_oracle":[0.4905681312084198,0.48838356137275696,0.02104838564991951]},{"id":"img_06","arm":"g448","question_id":"in_red_zone","oracle_top2_margin":0.017518073320388794,"argmax_equal":true,"max_abs_dp":0.0017277002334594727,"probs":[0.5104867219924927,0.4895133078098297],"probs_oracle":[0.5087590217590332,0.4912409484386444]},{"id":"photo_01","arm":"g256","question_id":"kinds","oracle_top2_margin":0.00908571481704712,"argmax_equal":true,"max_abs_dp":0.0013225078582763672,"probs":[0.02351478859782219,0.11496720463037491,0.38068902492523193,0.3916814923286438,0.08914753049612045],"probs_oracle":[0.023699797689914703,0.11542528122663498,0.3812732696533203,0.39035898447036743,0.08924264460802078]}],"timing":{"contended":true,"by_T":{"T<=300":{"runs":144,"median_tokens":190.0,"median_padded_T":512.0,"first_call_ms_median":6.262687486014329,"second_call_ms_median":6.048416995326988,"host_ms_median":0.28033349371980876},"300<T<=700":{"runs":59,"median_tokens":522.0,"median_padded_T":1024.0,"first_call_ms_median":7.967124984133989,"second_call_ms_median":7.652125001186505,"host_ms_median":0.671416986733675},"700<T<=1500":{"runs":7,"median_tokens":1038.0,"median_padded_T":2048.0,"first_call_ms_median":11.639791977358982,"second_call_ms_median":11.078334006015211,"host_ms_median":1.1001249949913472},"T>1500":{"runs":3,"median_tokens":2494.0,"median_padded_T":4096.0,"first_call_ms_median":19.852416997309774,"second_call_ms_median":18.95258401054889,"host_ms_median":3.829541994491592}},"all_first_call_ms_median":6.305333023192361,"all_second_call_ms_median":6.069875002140179},"processes":[{"pid":87246,"started":1791003246.738302,"load_model_seconds":0.005324625002685934,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08602091699140146,"t1024":0.0012139159953221679,"t2048":0.0011087080056313425,"t4096":0.0010673330107238144},"reset_check":{"run":["own_t01","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003247.574118,"process_wall_seconds":1.3154080839885864,"runs":36},{"pid":87276,"started":1791003248.037363,"load_model_seconds":0.005442165973363444,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08829541699378751,"t1024":0.001236415992025286,"t2048":0.004286792012862861,"t4096":0.001025042001856491},"reset_check":{"run":["img_03","native","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003248.876385,"process_wall_seconds":1.2959896249813028,"runs":36},{"pid":87319,"started":1791003249.352469,"load_model_seconds":0.005908666993491352,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08852587500587106,"t1024":0.0010990000155288726,"t2048":0.00103629200020805,"t4096":0.0010527500126045197},"reset_check":{"run":["semif_942ec65dac0bfb9874f9","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003250.000324,"process_wall_seconds":1.1213451249932405,"runs":36},{"pid":87333,"started":1791003250.4654489,"load_model_seconds":0.005436582985566929,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08369687499362044,"t1024":0.001224583014845848,"t2048":0.0011276250006631017,"t4096":0.001150957978097722},"reset_check":{"run":["semif_d0e36c4cb10a29b3191c","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003251.1051168,"process_wall_seconds":1.1037946669966914,"runs":36},{"pid":87349,"started":1791003251.572234,"load_model_seconds":0.005433290993096307,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08537512499606237,"t1024":0.0011938329844269902,"t2048":0.0010853339917957783,"t4096":0.0011329170083627105},"reset_check":{"run":["semif_f46f392ef9e9e9df564b","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003252.209228,"process_wall_seconds":1.1033866250072606,"runs":36},{"pid":87362,"started":1791003252.670305,"load_model_seconds":0.005298292002407834,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.08802020800067112,"t1024":0.0012366669834591448,"t2048":0.0011289590038359165,"t4096":0.004919583006994799},"reset_check":{"run":["semif_eafc22c8c40df3932a8e","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003253.278504,"process_wall_seconds":1.0668614169990178,"runs":33},{"pid":87378,"started":1791003253.737597,"load_model_seconds":0.005752083990955725,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.086052542028483,"t1024":0.0012305419950280339,"t2048":0.0011732919956557453,"t4096":0.0010628330055624247},"reset_check":{"run":["own_t01","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791003254.1284108,"process_wall_seconds":0.8497880829963833,"runs":16}],"red_arm":{"arms":{"span_shift_right_1":"q_avg and o_avg rolled one column right (every span one token later)","lexical_zero":"lexical = 0 for every option","member_roll_1":"member columns rotated by one option (option o counted under option o-1's question)"},"runs":[{"id":"own_t01","arm":"text","arm_name":"span_shift_right_1","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.0033146440982818604,"base_equals_gate_run":true},{"id":"own_t01","arm":"text","arm_name":"lexical_zero","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.05540060997009277,"base_equals_gate_run":true},{"id":"own_t01","arm":"text","arm_name":"member_roll_1","questions":3,"argmax_changed":1,"max_abs_dp_vs_base":0.10486787557601929,"base_equals_gate_run":true},{"id":"own_j08","arm":"text","arm_name":"span_shift_right_1","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.055809736251831055,"base_equals_gate_run":true},{"id":"own_j08","arm":"text","arm_name":"lexical_zero","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.017064779996871948,"base_equals_gate_run":true},{"id":"own_j08","arm":"text","arm_name":"member_roll_1","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.05064466595649719,"base_equals_gate_run":true},{"id":"img_01","arm":"g256","arm_name":"span_shift_right_1","questions":3,"argmax_changed":1,"max_abs_dp_vs_base":0.015392720699310303,"base_equals_gate_run":true},{"id":"img_01","arm":"g256","arm_name":"lexical_zero","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.029729098081588745,"base_equals_gate_run":true},{"id":"img_01","arm":"g256","arm_name":"member_roll_1","questions":3,"argmax_changed":2,"max_abs_dp_vs_base":0.3483545482158661,"base_equals_gate_run":true},{"id":"img_07","arm":"g448","arm_name":"span_shift_right_1","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.008411049842834473,"base_equals_gate_run":true},{"id":"img_07","arm":"g448","arm_name":"lexical_zero","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.03367370367050171,"base_equals_gate_run":true},{"id":"img_07","arm":"g448","arm_name":"member_roll_1","questions":3,"argmax_changed":0,"max_abs_dp_vs_base":0.05172383785247803,"base_equals_gate_run":true}],"by_arm":{"span_shift_right_1":{"runs":4,"argmax_changed_total":1,"questions_total":12,"max_abs_dp_vs_base":0.055809736251831055,"moves":false},"lexical_zero":{"runs":4,"argmax_changed_total":0,"questions_total":12,"max_abs_dp_vs_base":0.05540060997009277,"moves":false},"member_roll_1":{"runs":4,"argmax_changed_total":3,"questions_total":12,"max_abs_dp_vs_base":0.3483545482158661,"moves":true}},"result":"NOT RED on every run"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max"},"script":{"gate_head.py":"1d66d14945d23d15b80866799a6287a08fcca41cf95ff56ea41e418a8c0b5e0b","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d"},"seconds":{"gpu_processes":7.865610416018171},"generated_at":"2026-10-03T04:54:14+00:00","runs":[{"id":"own_t01","arm":"text","variant":"base","tokens":436,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":1.209333015140146,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0015636682510375977,"mean_abs_dp":0.00033182622864842416,"max_abs_dlogit":0.007732391357421875,"author_head_fp32_same_hidden":{"max_abs_dp":0.0015636980533599854,"mean_abs_dp":0.0003318349597975612},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t02","arm":"text","variant":"base","tokens":599,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,5],[5,7],[7,12],[12,14]],"host_ms":1.875291985925287,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012312531471252441,"mean_abs_dp":0.00022528563360018388,"max_abs_dlogit":0.00748133659362793,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012311935424804688,"mean_abs_dp":0.00022528863545241102},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t03","arm":"text","variant":"base","tokens":463,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":7,"layout":[[0,3],[3,5],[5,7]],"host_ms":0.5515000084415078,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00822708010673523,"mean_abs_dp":0.003257949171321733,"max_abs_dlogit":0.031670451164245605,"author_head_fp32_same_hidden":{"max_abs_dp":0.008227109909057617,"mean_abs_dp":0.0032579579523631503},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_t04","arm":"text","variant":"base","tokens":493,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,9],[9,11]],"host_ms":0.5560830177273601,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0003347955644130707,"mean_abs_dp":0.0001286946406418627,"max_abs_dlogit":0.012346416711807251,"author_head_fp32_same_hidden":{"max_abs_dp":0.0003347974270582199,"mean_abs_dp":0.0001286944289776412},"vs_author_head_same_hidden_max_abs_dp":9.313225746154785e-09,"argmax_equal_author_head_same_hidden":true},{"id":"own_t05","arm":"text","variant":"base","tokens":553,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,5],[5,7],[7,11],[11,13]],"host_ms":0.8160420111380517,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00026407837867736816,"mean_abs_dp":6.600426366695992e-05,"max_abs_dlogit":0.0061678290367126465,"author_head_fp32_same_hidden":{"max_abs_dp":0.00026410818099975586,"mean_abs_dp":6.600669943369352e-05},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_t06","arm":"text","variant":"base","tokens":411,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.5051659827586263,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005132555961608887,"mean_abs_dp":0.00015263316325015493,"max_abs_dlogit":0.008383870124816895,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005132555961608887,"mean_abs_dp":0.00015263331847058402},"vs_author_head_same_hidden_max_abs_dp":4.470348358154297e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_t07","arm":"text","variant":"base","tokens":438,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":12,"layout":[[0,6],[6,10],[10,12]],"host_ms":0.6019589782226831,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.006536126136779785,"mean_abs_dp":0.0013508718499603372,"max_abs_dlogit":0.05237102508544922,"author_head_fp32_same_hidden":{"max_abs_dp":0.006536126136779785,"mean_abs_dp":0.0013508485668959718},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t08","arm":"text","variant":"base","tokens":604,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,3],[3,5],[5,8],[8,18]],"host_ms":1.3190419995225966,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00021904706954956055,"mean_abs_dp":3.754263485057486e-05,"max_abs_dlogit":0.0068152546882629395,"author_head_fp32_same_hidden":{"max_abs_dp":0.00021904706954956055,"mean_abs_dp":3.7541703528000246e-05},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_t09","arm":"text","variant":"base","tokens":582,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,4],[4,6],[6,9],[9,16]],"host_ms":0.9103329794015735,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0014589130878448486,"mean_abs_dp":0.0002433915506117046,"max_abs_dlogit":0.04559612274169922,"author_head_fp32_same_hidden":{"max_abs_dp":0.0014589130878448486,"mean_abs_dp":0.00024339539231732488},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t10","arm":"text","variant":"base","tokens":396,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":8,"layout":[[0,2],[2,6],[6,8]],"host_ms":0.5013330082874745,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004936456680297852,"mean_abs_dp":0.00017888809088617563,"max_abs_dlogit":0.03242567181587219,"author_head_fp32_same_hidden":{"max_abs_dp":0.0004936456680297852,"mean_abs_dp":0.0001788836671039462},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_t11","arm":"text","variant":"base","tokens":497,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,2],[2,4],[4,8],[8,11]],"host_ms":0.671416986733675,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0008802413940429688,"mean_abs_dp":0.00022363324056972158,"max_abs_dlogit":0.013683438301086426,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008801817893981934,"mean_abs_dp":0.0002236412838101387},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t12","arm":"text","variant":"base","tokens":967,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":29,"layout":[[0,2],[2,8],[8,10],[10,13],[13,18],[18,23],[23,25],[25,29]],"host_ms":1.9417500006966293,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012277215719223022,"mean_abs_dp":0.0002732836994631537,"max_abs_dlogit":0.014318466186523438,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012277811765670776,"mean_abs_dp":0.00027329651317719753},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t13","arm":"text","variant":"base","tokens":370,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":8,"layout":[[0,2],[2,5],[5,8]],"host_ms":0.48012498882599175,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"own_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0008168071508407593,"mean_abs_dp":0.00022820383310317993,"max_abs_dlogit":0.018725872039794922,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008167773485183716,"mean_abs_dp":0.0002281981287524104},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_t14","arm":"text","variant":"base","tokens":2389,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":15,"layout":[[0,5],[5,7],[7,11],[11,15]],"host_ms":5.324375000782311,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"own_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.001191943883895874,"mean_abs_dp":0.00022992951174577076,"max_abs_dlogit":0.0149383544921875,"author_head_fp32_same_hidden":{"max_abs_dp":0.0011919289827346802,"mean_abs_dp":0.00022992355128129323},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t15","arm":"text","variant":"base","tokens":2603,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":22,"layout":[[0,5],[5,7],[7,11],[11,13],[13,18],[18,22]],"host_ms":3.829541994491592,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"own_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005500614643096924,"mean_abs_dp":7.559955966743556e-05,"max_abs_dlogit":0.011163830757141113,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005500614643096924,"mean_abs_dp":7.560548626563765e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_t16","arm":"text","variant":"base","tokens":2494,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":19,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17],[17,19]],"host_ms":2.0926249853800982,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"own_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0008679032325744629,"mean_abs_dp":0.00011320964825388633,"max_abs_dlogit":0.034476637840270996,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008678436279296875,"mean_abs_dp":0.00011320162785092467},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j01","arm":"text","variant":"base","tokens":476,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":0.5351250001695007,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012674182653427124,"mean_abs_dp":0.0003848218824714422,"max_abs_dlogit":0.0159376859664917,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012674480676651,"mean_abs_dp":0.0003848223015666008},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j02","arm":"text","variant":"base","tokens":475,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,2],[2,6],[6,8],[8,11]],"host_ms":0.5507919995579869,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0010565221309661865,"mean_abs_dp":0.00023182185197418386,"max_abs_dlogit":0.00964885950088501,"author_head_fp32_same_hidden":{"max_abs_dp":0.0010564625263214111,"mean_abs_dp":0.00023181516338478434},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j03","arm":"text","variant":"base","tokens":585,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,2],[2,7],[7,10],[10,14]],"host_ms":0.7614170026499778,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0018704235553741455,"mean_abs_dp":0.00033807421901396343,"max_abs_dlogit":0.009988546371459961,"author_head_fp32_same_hidden":{"max_abs_dp":0.0018704235553741455,"mean_abs_dp":0.000338081070887191},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j04","arm":"text","variant":"base","tokens":569,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,6],[6,11],[11,14]],"host_ms":0.6942919862922281,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.005557417869567871,"mean_abs_dp":0.0009066607869629349,"max_abs_dlogit":0.02545994520187378,"author_head_fp32_same_hidden":{"max_abs_dp":0.005557417869567871,"mean_abs_dp":0.0009066719295723098},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j05","arm":"text","variant":"base","tokens":539,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,11],[11,15]],"host_ms":0.7169589807745069,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0033428072929382324,"mean_abs_dp":0.0005667551265408596,"max_abs_dlogit":0.023497939109802246,"author_head_fp32_same_hidden":{"max_abs_dp":0.003342866897583008,"mean_abs_dp":0.0005667614595343669},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j06","arm":"text","variant":"base","tokens":616,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,6],[6,8],[8,12],[12,14],[14,17]],"host_ms":0.7918750052340329,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012636184692382812,"mean_abs_dp":0.00020470659193747183,"max_abs_dlogit":0.008612394332885742,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012635886669158936,"mean_abs_dp":0.0002046917866477195},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_j07","arm":"text","variant":"base","tokens":595,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,5],[5,10],[10,14],[14,16]],"host_ms":0.764124997658655,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012389570474624634,"mean_abs_dp":0.0002826069830916822,"max_abs_dlogit":0.03170323371887207,"author_head_fp32_same_hidden":{"max_abs_dp":0.001238986849784851,"mean_abs_dp":0.0002826033451128751},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j08","arm":"text","variant":"base","tokens":473,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.582000007852912,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0010389983654022217,"mean_abs_dp":0.0003438945859670639,"max_abs_dlogit":0.00778573751449585,"author_head_fp32_same_hidden":{"max_abs_dp":0.0010389387607574463,"mean_abs_dp":0.00034389309585094454},"vs_author_head_same_hidden_max_abs_dp":8.940696716308594e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j09","arm":"text","variant":"base","tokens":573,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":12,"layout":[[0,5],[5,7],[7,9],[9,12]],"host_ms":0.7001660123933107,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00140303373336792,"mean_abs_dp":0.0005376609818389019,"max_abs_dlogit":0.01997077465057373,"author_head_fp32_same_hidden":{"max_abs_dp":0.0014029145240783691,"mean_abs_dp":0.0005376191499332587},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"own_j10","arm":"text","variant":"base","tokens":413,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,4],[4,9],[9,11]],"host_ms":0.5140829889569432,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0008875727653503418,"mean_abs_dp":0.0003986296836625446,"max_abs_dlogit":0.0071707963943481445,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008876323699951172,"mean_abs_dp":0.0003986284136772156},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j11","arm":"text","variant":"base","tokens":463,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.48658400191925466,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0007089376449584961,"mean_abs_dp":0.0002444388810545206,"max_abs_dlogit":0.011611342430114746,"author_head_fp32_same_hidden":{"max_abs_dp":0.0007089376449584961,"mean_abs_dp":0.0002444382756948471},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"own_j12","arm":"text","variant":"base","tokens":562,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,2],[2,6],[6,10],[10,13]],"host_ms":0.7515829929616302,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"own_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012025237083435059,"mean_abs_dp":0.00039697237885915314,"max_abs_dlogit":0.006821393966674805,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012024641036987305,"mean_abs_dp":0.0003969523196036999},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_01","arm":"g256","variant":"base","tokens":430,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,6],[6,8],[8,11]],"host_ms":0.4972080059815198,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0013368427753448486,"mean_abs_dp":0.00045962995764884084,"max_abs_dlogit":0.014540314674377441,"author_head_fp32_same_hidden":{"max_abs_dp":0.001336812973022461,"mean_abs_dp":0.000459619289772077},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_01","arm":"g448","variant":"base","tokens":562,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,6],[6,8],[8,11]],"host_ms":0.60224998742342,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006278753280639648,"mean_abs_dp":0.0003032542087815025,"max_abs_dlogit":0.013925492763519287,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006278157234191895,"mean_abs_dp":0.0003032310950485143},"vs_author_head_same_hidden_max_abs_dp":1.341104507446289e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_01","arm":"native","variant":"base","tokens":430,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,6],[6,8],[8,11]],"host_ms":0.5067090096417814,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0013368427753448486,"mean_abs_dp":0.00045962995764884084,"max_abs_dlogit":0.014540314674377441,"author_head_fp32_same_hidden":{"max_abs_dp":0.001336812973022461,"mean_abs_dp":0.000459619289772077},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_02","arm":"g256","variant":"base","tokens":466,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,5],[5,7],[7,11]],"host_ms":0.5697919987142086,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005563646554946899,"mean_abs_dp":0.00015200640667568553,"max_abs_dlogit":0.005063533782958984,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005563795566558838,"mean_abs_dp":0.00015199794010682538},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_02","arm":"g448","variant":"base","tokens":598,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,5],[5,7],[7,11]],"host_ms":0.6318750092759728,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0007899552583694458,"mean_abs_dp":0.00022618895904584363,"max_abs_dlogit":0.01494908332824707,"author_head_fp32_same_hidden":{"max_abs_dp":0.0007899999618530273,"mean_abs_dp":0.0002262041142041033},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_02","arm":"native","variant":"base","tokens":702,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,5],[5,7],[7,11]],"host_ms":0.7377919973805547,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00045239925384521484,"mean_abs_dp":0.00013014733452688563,"max_abs_dlogit":0.010586142539978027,"author_head_fp32_same_hidden":{"max_abs_dp":0.00045245885848999023,"mean_abs_dp":0.0001301506364887411},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_03","arm":"g256","variant":"base","tokens":495,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":15,"layout":[[0,10],[10,12],[12,15]],"host_ms":0.6157500029075891,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0007660388946533203,"mean_abs_dp":0.00019937902688980103,"max_abs_dlogit":0.012104988098144531,"author_head_fp32_same_hidden":{"max_abs_dp":0.0007659792900085449,"mean_abs_dp":0.0001993585377931595},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_03","arm":"g448","variant":"base","tokens":627,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,10],[10,12],[12,15]],"host_ms":0.7003339997027069,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006858110427856445,"mean_abs_dp":0.00017027907694379488,"max_abs_dlogit":0.010346770286560059,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006859302520751953,"mean_abs_dp":0.0001702719988922278},"vs_author_head_same_hidden_max_abs_dp":1.7881393432617188e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_03","arm":"native","variant":"base","tokens":501,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":15,"layout":[[0,10],[10,12],[12,15]],"host_ms":1.3153330073691905,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004870891571044922,"mean_abs_dp":0.00012751587977011998,"max_abs_dlogit":0.013212919235229492,"author_head_fp32_same_hidden":{"max_abs_dp":0.0004870295524597168,"mean_abs_dp":0.00012751339624325435},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_04","arm":"g256","variant":"base","tokens":558,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,12],[12,15]],"host_ms":1.6791250091046095,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0002625584602355957,"mean_abs_dp":0.00010022207473715146,"max_abs_dlogit":0.014873862266540527,"author_head_fp32_same_hidden":{"max_abs_dp":0.0002624392509460449,"mean_abs_dp":0.00010021145765980085},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_04","arm":"g448","variant":"base","tokens":690,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,12],[12,15]],"host_ms":1.2579589965753257,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0003916621208190918,"mean_abs_dp":0.00010014787937204042,"max_abs_dlogit":0.00978851318359375,"author_head_fp32_same_hidden":{"max_abs_dp":0.0003916621208190918,"mean_abs_dp":0.00010013872136672338},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_04","arm":"native","variant":"base","tokens":1262,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,6],[6,12],[12,15]],"host_ms":3.2067499996628612,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005682706832885742,"mean_abs_dp":0.0001020798459649086,"max_abs_dlogit":0.016565561294555664,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005683302879333496,"mean_abs_dp":0.00010209931060671807},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_05","arm":"g256","variant":"base","tokens":464,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.5362500087358057,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00013369321823120117,"mean_abs_dp":5.030869082971053e-05,"max_abs_dlogit":0.0020380020141601562,"author_head_fp32_same_hidden":{"max_abs_dp":0.00013381242752075195,"mean_abs_dp":5.033400587060235e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_05","arm":"g448","variant":"base","tokens":596,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.6382919964380562,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00019890069961547852,"mean_abs_dp":7.904464886947112e-05,"max_abs_dlogit":0.0034530162811279297,"author_head_fp32_same_hidden":{"max_abs_dp":0.00019890069961547852,"mean_abs_dp":7.903008637103167e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_05","arm":"native","variant":"base","tokens":464,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.53645801381208,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00013369321823120117,"mean_abs_dp":5.030869082971053e-05,"max_abs_dlogit":0.0020380020141601562,"author_head_fp32_same_hidden":{"max_abs_dp":0.00013381242752075195,"mean_abs_dp":5.033400587060235e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_06","arm":"g256","variant":"base","tokens":465,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.7049590058159083,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0014894604682922363,"mean_abs_dp":0.0007030891254544258,"max_abs_dlogit":0.019919991493225098,"author_head_fp32_same_hidden":{"max_abs_dp":0.0014894604682922363,"mean_abs_dp":0.0007030850276350975},"vs_author_head_same_hidden_max_abs_dp":8.940696716308594e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_06","arm":"g448","variant":"base","tokens":597,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.7813749834895134,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.002756863832473755,"mean_abs_dp":0.0011138010770082474,"max_abs_dlogit":0.024670958518981934,"author_head_fp32_same_hidden":{"max_abs_dp":0.002757042646408081,"mean_abs_dp":0.0011138321831822395},"vs_author_head_same_hidden_max_abs_dp":1.7881393432617188e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_06","arm":"native","variant":"base","tokens":471,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,2],[2,6],[6,10]],"host_ms":0.6644579989369959,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012745857238769531,"mean_abs_dp":0.0005537913180887699,"max_abs_dlogit":0.01906609535217285,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012744665145874023,"mean_abs_dp":0.000553762074559927},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_07","arm":"g256","variant":"base","tokens":426,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.6638749910052866,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.011243820190429688,"mean_abs_dp":0.004165478143841028,"max_abs_dlogit":0.05287814140319824,"author_head_fp32_same_hidden":{"max_abs_dp":0.011243879795074463,"mean_abs_dp":0.004165483918040991},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_07","arm":"g448","variant":"base","tokens":558,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.7305420003831387,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00047403573989868164,"mean_abs_dp":0.00012602685019373893,"max_abs_dlogit":0.004275918006896973,"author_head_fp32_same_hidden":{"max_abs_dp":0.00047403573989868164,"mean_abs_dp":0.00012602005153894424},"vs_author_head_same_hidden_max_abs_dp":6.705522537231445e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_07","arm":"native","variant":"base","tokens":662,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,8],[8,10]],"host_ms":0.6947920192033052,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00027978047728538513,"mean_abs_dp":7.085399702191352e-05,"max_abs_dlogit":0.007892608642578125,"author_head_fp32_same_hidden":{"max_abs_dp":0.0002797357738018036,"mean_abs_dp":7.084468379616737e-05},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_08","arm":"g256","variant":"base","tokens":522,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,9],[9,11],[11,14]],"host_ms":0.741625000955537,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0014191269874572754,"mean_abs_dp":0.0002739187330007553,"max_abs_dlogit":0.016761183738708496,"author_head_fp32_same_hidden":{"max_abs_dp":0.0014191865921020508,"mean_abs_dp":0.0002739301084407738},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_08","arm":"g448","variant":"base","tokens":654,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,9],[9,11],[11,14]],"host_ms":0.7220830011647195,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00043016672134399414,"mean_abs_dp":0.00013055718903030668,"max_abs_dlogit":0.0065631866455078125,"author_head_fp32_same_hidden":{"max_abs_dp":0.00043016672134399414,"mean_abs_dp":0.0001305408908852509},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_08","arm":"native","variant":"base","tokens":1226,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,9],[9,11],[11,14]],"host_ms":1.1739580077119172,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.001103043556213379,"mean_abs_dp":0.0002860772157354014,"max_abs_dlogit":0.012650251388549805,"author_head_fp32_same_hidden":{"max_abs_dp":0.0011031031608581543,"mean_abs_dp":0.00028607169432299476},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"img_09","arm":"g256","variant":"base","tokens":460,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.5572910013142973,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.001811981201171875,"mean_abs_dp":0.0003417654132301157,"max_abs_dlogit":0.008774757385253906,"author_head_fp32_same_hidden":{"max_abs_dp":0.0018119215965270996,"mean_abs_dp":0.00034175576134161514},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_09","arm":"g448","variant":"base","tokens":592,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.6837910041213036,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005584061145782471,"mean_abs_dp":0.00010891385715116154,"max_abs_dlogit":0.009239435195922852,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005584359169006348,"mean_abs_dp":0.00010892096906900406},"vs_author_head_same_hidden_max_abs_dp":4.470348358154297e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_09","arm":"native","variant":"base","tokens":696,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":11,"layout":[[0,3],[3,6],[6,8],[8,11]],"host_ms":0.7175829960033298,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00023049116134643555,"mean_abs_dp":7.358371195468035e-05,"max_abs_dlogit":0.005457878112792969,"author_head_fp32_same_hidden":{"max_abs_dp":0.00023055076599121094,"mean_abs_dp":7.358972321857105e-05},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_10","arm":"g256","variant":"base","tokens":405,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.5207909853197634,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0001195669174194336,"mean_abs_dp":5.325804361038738e-05,"max_abs_dlogit":0.007907629013061523,"author_head_fp32_same_hidden":{"max_abs_dp":0.00011952221393585205,"mean_abs_dp":5.325395613908768e-05},"vs_author_head_same_hidden_max_abs_dp":6.332993507385254e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_10","arm":"g448","variant":"base","tokens":537,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.6341250264085829,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00024192780256271362,"mean_abs_dp":0.00010469167803724606,"max_abs_dlogit":0.007996082305908203,"author_head_fp32_same_hidden":{"max_abs_dp":0.00024192780256271362,"mean_abs_dp":0.00010469033279352718},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"img_10","arm":"native","variant":"base","tokens":405,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":9,"layout":[[0,4],[4,6],[6,9]],"host_ms":0.5297499883454293,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"own_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00017770379781723022,"mean_abs_dp":8.096452802419662e-05,"max_abs_dlogit":0.01063382625579834,"author_head_fp32_same_hidden":{"max_abs_dp":0.00017774850130081177,"mean_abs_dp":8.097528997394774e-05},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"photo_01","arm":"g256","variant":"base","tokens":556,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,6],[6,8],[8,13],[13,16]],"host_ms":0.7986249984242022,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0037543773651123047,"mean_abs_dp":0.000650876434519887,"max_abs_dlogit":0.01890707015991211,"author_head_fp32_same_hidden":{"max_abs_dp":0.003754258155822754,"mean_abs_dp":0.000650860951282084},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"photo_01","arm":"g448","variant":"base","tokens":688,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,6],[6,8],[8,13],[13,16]],"host_ms":0.8210829982999712,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.005205422639846802,"mean_abs_dp":0.000885498826391995,"max_abs_dlogit":0.016104936599731445,"author_head_fp32_same_hidden":{"max_abs_dp":0.005205392837524414,"mean_abs_dp":0.0008855023188516498},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"photo_02","arm":"g256","variant":"base","tokens":499,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,8],[8,11],[11,13]],"host_ms":0.650041998596862,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0013578534126281738,"mean_abs_dp":0.00022693773588308922,"max_abs_dlogit":0.006471395492553711,"author_head_fp32_same_hidden":{"max_abs_dp":0.0013578534126281738,"mean_abs_dp":0.00022694747895002365},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"photo_02","arm":"g448","variant":"base","tokens":631,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,8],[8,11],[11,13]],"host_ms":0.7375419954769313,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005651712417602539,"mean_abs_dp":0.00011622440069913864,"max_abs_dlogit":0.0035223960876464844,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005651116371154785,"mean_abs_dp":0.0001162282692698332},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"photo_02","arm":"native","variant":"base","tokens":1035,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,8],[8,11],[11,13]],"host_ms":1.1001249949913472,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00014072656631469727,"mean_abs_dp":4.566159959022816e-05,"max_abs_dlogit":0.005042672157287598,"author_head_fp32_same_hidden":{"max_abs_dp":0.00014072656631469727,"mean_abs_dp":4.5664644298645166e-05},"vs_author_head_same_hidden_max_abs_dp":4.0978193283081055e-08,"argmax_equal_author_head_same_hidden":true},{"id":"photo_03","arm":"g256","variant":"base","tokens":505,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,4],[4,8],[8,12]],"host_ms":0.6126250082161278,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.988380432128906e-05,"mean_abs_dp":3.342741789917151e-05,"max_abs_dlogit":0.004330873489379883,"author_head_fp32_same_hidden":{"max_abs_dp":8.988380432128906e-05,"mean_abs_dp":3.343044469753901e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"photo_03","arm":"g448","variant":"base","tokens":637,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,4],[4,8],[8,12]],"host_ms":0.7630829932168126,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00023186951875686646,"mean_abs_dp":7.754160712162654e-05,"max_abs_dlogit":0.005363941192626953,"author_head_fp32_same_hidden":{"max_abs_dp":0.00023186206817626953,"mean_abs_dp":7.75544128070275e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"photo_03","arm":"native","variant":"base","tokens":1041,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,4],[4,8],[8,12]],"host_ms":1.0287500044796616,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00018852949142456055,"mean_abs_dp":8.01204393307368e-05,"max_abs_dlogit":0.011218070983886719,"author_head_fp32_same_hidden":{"max_abs_dp":0.00018852949142456055,"mean_abs_dp":8.013464199999969e-05},"vs_author_head_same_hidden_max_abs_dp":6.332993507385254e-08,"argmax_equal_author_head_same_hidden":true},{"id":"photo_04","arm":"g256","variant":"base","tokens":502,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":0.5719169857911766,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000141143798828125,"mean_abs_dp":3.2923005234736665e-05,"max_abs_dlogit":0.010097026824951172,"author_head_fp32_same_hidden":{"max_abs_dp":0.000141143798828125,"mean_abs_dp":3.292440221859859e-05},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"photo_04","arm":"g448","variant":"base","tokens":634,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":0.7794169941917062,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00010961294174194336,"mean_abs_dp":2.9135531244369653e-05,"max_abs_dlogit":0.004208207130432129,"author_head_fp32_same_hidden":{"max_abs_dp":0.00010961294174194336,"mean_abs_dp":2.9125680717138144e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"photo_04","arm":"native","variant":"base","tokens":1038,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":1.088457996957004,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"cc0_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00010752677917480469,"mean_abs_dp":3.9093447132752495e-05,"max_abs_dlogit":0.004675313830375671,"author_head_fp32_same_hidden":{"max_abs_dp":0.0001074075698852539,"mean_abs_dp":3.908176978047077e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_a3f18f3a63d45345942b","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.292791984975338,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.4199485778808594e-05,"mean_abs_dp":1.615658402442932e-05,"max_abs_dlogit":0.0005602836608886719,"author_head_fp32_same_hidden":{"max_abs_dp":2.4199485778808594e-05,"mean_abs_dp":1.615099608898163e-05},"vs_author_head_same_hidden_max_abs_dp":1.6763806343078613e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_f40beba9088c8db8bbd6","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2748749975580722,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006181001663208008,"mean_abs_dp":0.00041206181049346924,"max_abs_dlogit":0.004513263702392578,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006181001663208008,"mean_abs_dp":0.00041206181049346924},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_0b43ea8e24e74d621f1b","arm":"text","variant":"base","tokens":177,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26470801094546914,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00012189149856567383,"mean_abs_dp":8.125106493631999e-05,"max_abs_dlogit":0.0018939971923828125,"author_head_fp32_same_hidden":{"max_abs_dp":0.00012189149856567383,"mean_abs_dp":8.12460978825887e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_942ec65dac0bfb9874f9","arm":"text","variant":"base","tokens":177,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.7975840126164258,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000499427318572998,"mean_abs_dp":0.00033293478190898895,"max_abs_dlogit":0.006093502044677734,"author_head_fp32_same_hidden":{"max_abs_dp":0.0004995465278625488,"mean_abs_dp":0.0003329856942097346},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_e0c140e9222d1bce5647","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.445583020336926,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.3890286684036255e-05,"mean_abs_dp":1.5913198391596477e-05,"max_abs_dlogit":0.002421855926513672,"author_head_fp32_same_hidden":{"max_abs_dp":2.392008900642395e-05,"mean_abs_dp":1.595790187517802e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_8e8c3804a3c15ebb31e3","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2789999998640269,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006719157099723816,"mean_abs_dp":0.00044794318576653797,"max_abs_dlogit":0.007906198501586914,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006719306111335754,"mean_abs_dp":0.0004479667792717616},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_56c419b12cd37cbe790c","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.281625019852072,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00016129016876220703,"mean_abs_dp":0.00010754416386286418,"max_abs_dlogit":0.003068685531616211,"author_head_fp32_same_hidden":{"max_abs_dp":0.00016129016876220703,"mean_abs_dp":0.00010754416386286418},"vs_author_head_same_hidden_max_abs_dp":0.0,"argmax_equal_author_head_same_hidden":true},{"id":"semif_75b470586834df8c568d","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2856670180335641,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006379485130310059,"mean_abs_dp":0.0004253070801496506,"max_abs_dlogit":0.005024433135986328,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006379485130310059,"mean_abs_dp":0.00042531266808509827},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_834f3b3e10cc61b33d25","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2767499827314168,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.70280647277832e-05,"mean_abs_dp":3.131603201230367e-05,"max_abs_dlogit":0.001324295997619629,"author_head_fp32_same_hidden":{"max_abs_dp":4.70280647277832e-05,"mean_abs_dp":3.131665289402008e-05},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_76c7a79b1ff2363a60c5","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2774579916149378,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.662441253662109e-06,"mean_abs_dp":3.8106615344683328e-06,"max_abs_dlogit":0.0011467933654785156,"author_head_fp32_same_hidden":{"max_abs_dp":5.662441253662109e-06,"mean_abs_dp":3.8106615344683328e-06},"vs_author_head_same_hidden_max_abs_dp":0.0,"argmax_equal_author_head_same_hidden":true},{"id":"semif_36c71bb649b7d5eccb1f","arm":"text","variant":"base","tokens":174,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28787500923499465,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.18353146314621e-06,"mean_abs_dp":5.4426491260528564e-06,"max_abs_dlogit":0.0018029212951660156,"author_head_fp32_same_hidden":{"max_abs_dp":8.18353146314621e-06,"mean_abs_dp":5.444822212060292e-06},"vs_author_head_same_hidden_max_abs_dp":6.51925802230835e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_eac5121a4a1bf391a08c","arm":"text","variant":"base","tokens":173,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26158400578424335,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.2040138244628906e-05,"mean_abs_dp":8.001302679379782e-06,"max_abs_dlogit":0.0005307197570800781,"author_head_fp32_same_hidden":{"max_abs_dp":1.2040138244628906e-05,"mean_abs_dp":7.99944003423055e-06},"vs_author_head_same_hidden_max_abs_dp":6.51925802230835e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_44574c44e243ee1b2f52","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26933400658890605,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.989738464355469e-05,"mean_abs_dp":6.660384436448415e-05,"max_abs_dlogit":0.0032987594604492188,"author_head_fp32_same_hidden":{"max_abs_dp":9.989738464355469e-05,"mean_abs_dp":6.659577290217082e-05},"vs_author_head_same_hidden_max_abs_dp":1.30385160446167e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_2608370d01783735c3ea","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28895799187012017,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00013047456741333008,"mean_abs_dp":8.70165725549062e-05,"max_abs_dlogit":0.0017213821411132812,"author_head_fp32_same_hidden":{"max_abs_dp":0.00013047456741333008,"mean_abs_dp":8.700912197430928e-05},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_ec94ef909f7918695ab3","arm":"text","variant":"base","tokens":180,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25733301299624145,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.264448165893555e-05,"mean_abs_dp":4.176671306292216e-05,"max_abs_dlogit":0.0013685226440429688,"author_head_fp32_same_hidden":{"max_abs_dp":6.276369094848633e-05,"mean_abs_dp":4.1816383600234985e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_03d919fd0e9b3a7cd1e6","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27129199588671327,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00016380101442337036,"mean_abs_dp":0.00010921061038970947,"max_abs_dlogit":0.002506256103515625,"author_head_fp32_same_hidden":{"max_abs_dp":0.0001637674868106842,"mean_abs_dp":0.00010916093985239665},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_86a15618ba4cb90a307e","arm":"text","variant":"base","tokens":179,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30691700521856546,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.002375304698944092,"mean_abs_dp":0.0015835165977478027,"max_abs_dlogit":0.018954992294311523,"author_head_fp32_same_hidden":{"max_abs_dp":0.002375364303588867,"mean_abs_dp":0.0015835513671239216},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_3dd9c3ac98f8ddf7bdd3","arm":"text","variant":"base","tokens":179,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29624998569488525,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00016170740127563477,"mean_abs_dp":0.0001077918956677119,"max_abs_dlogit":0.002460956573486328,"author_head_fp32_same_hidden":{"max_abs_dp":0.00016170740127563477,"mean_abs_dp":0.00010778382420539856},"vs_author_head_same_hidden_max_abs_dp":2.421438694000244e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_36273dd4777876a26338","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2995000104419887,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.611513137817383e-05,"mean_abs_dp":5.0747767090797424e-05,"max_abs_dlogit":0.0010429024696350098,"author_head_fp32_same_hidden":{"max_abs_dp":7.611513137817383e-05,"mean_abs_dp":5.074838797251383e-05},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_07ee70cfe4b01330ce89","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2884170098695904,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000603020191192627,"mean_abs_dp":0.0004019998013973236,"max_abs_dlogit":0.006039261817932129,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006030797958374023,"mean_abs_dp":0.0004020469884077708},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_51ceab3ce622c7c759f1","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4301670123822987,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.918741226196289e-05,"mean_abs_dp":3.944026927153269e-05,"max_abs_dlogit":0.0037097930908203125,"author_head_fp32_same_hidden":{"max_abs_dp":5.906820297241211e-05,"mean_abs_dp":3.938749432563782e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b8091be929b1e489fcc4","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25716700474731624,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004099607467651367,"mean_abs_dp":0.00027332579096158344,"max_abs_dlogit":0.006588578224182129,"author_head_fp32_same_hidden":{"max_abs_dp":0.0004099607467651367,"mean_abs_dp":0.00027333199977874756},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_c89b86e3f55123bfa506","arm":"text","variant":"base","tokens":182,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2611250092741102,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.7507543563842773e-05,"mean_abs_dp":1.8344571193059284e-05,"max_abs_dlogit":0.0004887580871582031,"author_head_fp32_same_hidden":{"max_abs_dp":2.750009298324585e-05,"mean_abs_dp":1.8343329429626465e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_3b0958cabc339005cf66","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27433299692347646,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00018423795700073242,"mean_abs_dp":0.0001227923979361852,"max_abs_dlogit":0.005845069885253906,"author_head_fp32_same_hidden":{"max_abs_dp":0.00018423795700073242,"mean_abs_dp":0.00012279177705446878},"vs_author_head_same_hidden_max_abs_dp":1.862645149230957e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_f2b4ec4930322fe45118","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.46620899229310453,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006383657455444336,"mean_abs_dp":0.00042557281752427417,"max_abs_dlogit":0.013895273208618164,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006383657455444336,"mean_abs_dp":0.0004255821307500203},"vs_author_head_same_hidden_max_abs_dp":2.421438694000244e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_61ac20b07a5189231e15","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4335420089773834,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0007973909378051758,"mean_abs_dp":0.0005316144476334254,"max_abs_dlogit":0.008166670799255371,"author_head_fp32_same_hidden":{"max_abs_dp":0.0007975101470947266,"mean_abs_dp":0.0005316610137621561},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4b3d794c004566610181","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.32983298297040164,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00038182735443115234,"mean_abs_dp":0.000254549706975619,"max_abs_dlogit":0.00818932056427002,"author_head_fp32_same_hidden":{"max_abs_dp":0.00038182735443115234,"mean_abs_dp":0.0002545534322659175},"vs_author_head_same_hidden_max_abs_dp":1.862645149230957e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b7c4e62d07986286fce0","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3044159966520965,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0053250789642333984,"mean_abs_dp":0.003550062576929728,"max_abs_dlogit":0.08375978469848633,"author_head_fp32_same_hidden":{"max_abs_dp":0.005324959754943848,"mean_abs_dp":0.0035499930381774902},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_9473603b44af63beab79","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29741600155830383,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0001342296600341797,"mean_abs_dp":8.947402238845825e-05,"max_abs_dlogit":0.007631063461303711,"author_head_fp32_same_hidden":{"max_abs_dp":0.0001342296600341797,"mean_abs_dp":8.94771267970403e-05},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_d41072dc35d8cd14c056","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.291750009637326,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.3048133850097656e-05,"mean_abs_dp":3.534431258837382e-05,"max_abs_dlogit":0.0009561777114868164,"author_head_fp32_same_hidden":{"max_abs_dp":5.3048133850097656e-05,"mean_abs_dp":3.534244994322459e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_33d6e5da58f0fee2490d","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28645899146795273,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.252527236938477e-05,"mean_abs_dp":4.166302581628164e-05,"max_abs_dlogit":0.006810665130615234,"author_head_fp32_same_hidden":{"max_abs_dp":6.252527236938477e-05,"mean_abs_dp":4.166302581628164e-05},"vs_author_head_same_hidden_max_abs_dp":5.587935447692871e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_823bf736b92e9d8fa66d","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29545798315666616,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005645155906677246,"mean_abs_dp":0.0003763437271118164,"max_abs_dlogit":0.010567665100097656,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005645155906677246,"mean_abs_dp":0.0003763313094774882},"vs_author_head_same_hidden_max_abs_dp":3.725290298461914e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_8a4c3de28be95c105ca1","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27749998844228685,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.003479182720184326,"mean_abs_dp":0.0023194526632626853,"max_abs_dlogit":0.05240601301193237,"author_head_fp32_same_hidden":{"max_abs_dp":0.0034791231155395508,"mean_abs_dp":0.002319390575091044},"vs_author_head_same_hidden_max_abs_dp":6.705522537231445e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_ea7ec66a5e8eac886548","arm":"text","variant":"base","tokens":192,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3178750048391521,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004500746726989746,"mean_abs_dp":0.00030002494653066,"max_abs_dlogit":0.009470939636230469,"author_head_fp32_same_hidden":{"max_abs_dp":0.00044995546340942383,"mean_abs_dp":0.0002999889353911082},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_25172cd6ddc49ab0ca70","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27779198717325926,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.298798739910126e-05,"mean_abs_dp":2.8628235061963398e-05,"max_abs_dlogit":0.004192829132080078,"author_head_fp32_same_hidden":{"max_abs_dp":4.301033914089203e-05,"mean_abs_dp":2.865058680375417e-05},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_fb73a85d386cf59b0c51","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27725001564249396,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.004906117916107178,"mean_abs_dp":0.003270761420329412,"max_abs_dlogit":0.02344059944152832,"author_head_fp32_same_hidden":{"max_abs_dp":0.004906117916107178,"mean_abs_dp":0.0032707912226517997},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_a02fd582d2881cc462b2","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2770000137388706,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.492273390293121e-05,"mean_abs_dp":2.327871819337209e-05,"max_abs_dlogit":0.006656169891357422,"author_head_fp32_same_hidden":{"max_abs_dp":3.493949770927429e-05,"mean_abs_dp":2.3284306128819782e-05},"vs_author_head_same_hidden_max_abs_dp":1.6763806343078613e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_19f37d5a97034bd7d04f","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28483301866799593,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00015783309936523438,"mean_abs_dp":0.00010521213213602702,"max_abs_dlogit":0.006381630897521973,"author_head_fp32_same_hidden":{"max_abs_dp":0.00015783309936523438,"mean_abs_dp":0.00010520840684572856},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_fda0ca94652c6d739ecf","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30754099134355783,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.827976226806641e-06,"mean_abs_dp":3.2372772693634033e-06,"max_abs_dlogit":0.00133514404296875,"author_head_fp32_same_hidden":{"max_abs_dp":4.827976226806641e-06,"mean_abs_dp":3.2372772693634033e-06},"vs_author_head_same_hidden_max_abs_dp":0.0,"argmax_equal_author_head_same_hidden":true},{"id":"semif_d0e36c4cb10a29b3191c","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.8405830012634397,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0018828511238098145,"mean_abs_dp":0.0012552415331204732,"max_abs_dlogit":0.0130690336227417,"author_head_fp32_same_hidden":{"max_abs_dp":0.0018828511238098145,"mean_abs_dp":0.001255229115486145},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_a0a39ace7e5337d038a7","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4465829988475889,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.1801719665527344e-05,"mean_abs_dp":7.828697562217712e-06,"max_abs_dlogit":0.0010294914245605469,"author_head_fp32_same_hidden":{"max_abs_dp":1.1801719665527344e-05,"mean_abs_dp":7.836148142814636e-06},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_eeae283841ce75dd25bf","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3137500025331974,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.111734986305237e-05,"mean_abs_dp":2.074489990870158e-05,"max_abs_dlogit":0.004038333892822266,"author_head_fp32_same_hidden":{"max_abs_dp":3.1109899282455444e-05,"mean_abs_dp":2.0739932854970295e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_00bb4cc043f7d2b288d4","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3082499897573143,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.697008550167084e-05,"mean_abs_dp":5.127800007661184e-05,"max_abs_dlogit":0.006481647491455078,"author_head_fp32_same_hidden":{"max_abs_dp":7.69738107919693e-05,"mean_abs_dp":5.128048360347748e-05},"vs_author_head_same_hidden_max_abs_dp":3.725290298461914e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_1a81ead69e93e5d0c86a","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30541702290065587,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.004739850759506226,"mean_abs_dp":0.003159940242767334,"max_abs_dlogit":0.06997990608215332,"author_head_fp32_same_hidden":{"max_abs_dp":0.004739880561828613,"mean_abs_dp":0.0031599452098210654},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4444dad25e438f128673","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3002499870490283,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00011444091796875,"mean_abs_dp":7.630946735541026e-05,"max_abs_dlogit":0.0027604103088378906,"author_head_fp32_same_hidden":{"max_abs_dp":0.00011444091796875,"mean_abs_dp":7.630946735541026e-05},"vs_author_head_same_hidden_max_abs_dp":0.0,"argmax_equal_author_head_same_hidden":true},{"id":"semif_3226202ba1ada2570e9f","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28137501794844866,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004112720489501953,"mean_abs_dp":0.0002742074429988861,"max_abs_dlogit":0.013236045837402344,"author_head_fp32_same_hidden":{"max_abs_dp":0.00041130930185317993,"mean_abs_dp":0.00027421116828918457},"vs_author_head_same_hidden_max_abs_dp":3.725290298461914e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_3f783eec5c90812479bc","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2825829724315554,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000321805477142334,"mean_abs_dp":0.0002145376056432724,"max_abs_dlogit":0.013304471969604492,"author_head_fp32_same_hidden":{"max_abs_dp":0.000321805477142334,"mean_abs_dp":0.00021453946828842163},"vs_author_head_same_hidden_max_abs_dp":5.587935447692871e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_ffee861474a9665a093a","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27116600540466607,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0021082162857055664,"mean_abs_dp":0.0014054377873738606,"max_abs_dlogit":0.022643566131591797,"author_head_fp32_same_hidden":{"max_abs_dp":0.0021080374717712402,"mean_abs_dp":0.0014053235451380413},"vs_author_head_same_hidden_max_abs_dp":1.7881393432617188e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_8c992e17e4fb54b044f5","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2600839943625033,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.21540641784668e-05,"mean_abs_dp":3.478489816188812e-05,"max_abs_dlogit":0.0028737783432006836,"author_head_fp32_same_hidden":{"max_abs_dp":5.21540641784668e-05,"mean_abs_dp":3.4786139925320945e-05},"vs_author_head_same_hidden_max_abs_dp":1.30385160446167e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_695dd61195ca025c4f4c","arm":"text","variant":"base","tokens":179,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28308399487286806,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.1696090698242188e-05,"mean_abs_dp":1.4468406637509664e-05,"max_abs_dlogit":0.0020275115966796875,"author_head_fp32_same_hidden":{"max_abs_dp":2.1696090698242188e-05,"mean_abs_dp":1.4468406637509664e-05},"vs_author_head_same_hidden_max_abs_dp":0.0,"argmax_equal_author_head_same_hidden":true},{"id":"semif_e6155431ab7b0f8170d4","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.33483398146927357,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.044120967388153e-05,"mean_abs_dp":2.0289793610572815e-05,"max_abs_dlogit":0.004231452941894531,"author_head_fp32_same_hidden":{"max_abs_dp":3.0436553061008453e-05,"mean_abs_dp":2.029258757829666e-05},"vs_author_head_same_hidden_max_abs_dp":1.30385160446167e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b49ecda75086d8c68d2a","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30370900640264153,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0010090731084346771,"mean_abs_dp":0.0006727017462253571,"max_abs_dlogit":0.10230028629302979,"author_head_fp32_same_hidden":{"max_abs_dp":0.0010090470314025879,"mean_abs_dp":0.0006727029879887899},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_587ced261c6092cd1325","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2703749923966825,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.378225326538086e-05,"mean_abs_dp":1.5862596531709034e-05,"max_abs_dlogit":0.0009226799011230469,"author_head_fp32_same_hidden":{"max_abs_dp":2.378225326538086e-05,"mean_abs_dp":1.586725314458211e-05},"vs_author_head_same_hidden_max_abs_dp":9.313225746154785e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_533d4423d311de82b27d","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27749998844228685,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.5914440155029297e-05,"mean_abs_dp":1.0584481060504913e-05,"max_abs_dlogit":0.0006062984466552734,"author_head_fp32_same_hidden":{"max_abs_dp":1.5914440155029297e-05,"mean_abs_dp":1.0591310759385427e-05},"vs_author_head_same_hidden_max_abs_dp":1.3969838619232178e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_2bf540d515a51755b25c","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.36087498301640153,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.267692565917969e-05,"mean_abs_dp":2.8438555697600048e-05,"max_abs_dlogit":0.0038633346557617188,"author_head_fp32_same_hidden":{"max_abs_dp":4.267692565917969e-05,"mean_abs_dp":2.844228098789851e-05},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_fdb9fc214aea2710763a","arm":"text","variant":"base","tokens":168,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2695829898584634,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0009981989860534668,"mean_abs_dp":0.0006654411554336548,"max_abs_dlogit":0.01687002182006836,"author_head_fp32_same_hidden":{"max_abs_dp":0.000998079776763916,"mean_abs_dp":0.000665405144294103},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_f6d68126b43f8c9ed183","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2810409932862967,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.3990869522094727e-06,"mean_abs_dp":1.5813857316970825e-06,"max_abs_dlogit":0.002147674560546875,"author_head_fp32_same_hidden":{"max_abs_dp":2.4121254682540894e-06,"mean_abs_dp":1.6205012798309326e-06},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_44bae0c8a1615a096c43","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25775001267902553,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.01865690946579e-06,"mean_abs_dp":2.6775524020195007e-06,"max_abs_dlogit":0.0006594657897949219,"author_head_fp32_same_hidden":{"max_abs_dp":4.01865690946579e-06,"mean_abs_dp":2.6775524020195007e-06},"vs_author_head_same_hidden_max_abs_dp":0.0,"argmax_equal_author_head_same_hidden":true},{"id":"semif_2175379f4c69e6207626","arm":"text","variant":"base","tokens":175,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2727090031839907,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00016289949417114258,"mean_abs_dp":0.00010856365164120992,"max_abs_dlogit":0.0030547380447387695,"author_head_fp32_same_hidden":{"max_abs_dp":0.00016289949417114258,"mean_abs_dp":0.00010856365164120992},"vs_author_head_same_hidden_max_abs_dp":3.725290298461914e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_def9fe59d262357a673a","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2745420206338167,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0003795623779296875,"mean_abs_dp":0.00025300805767377216,"max_abs_dlogit":0.0035810470581054688,"author_head_fp32_same_hidden":{"max_abs_dp":0.0003795623779296875,"mean_abs_dp":0.00025299439827601117},"vs_author_head_same_hidden_max_abs_dp":2.60770320892334e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_712f6ffb763eef7c9a12","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2632079995237291,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":5.519390106201172e-05,"mean_abs_dp":3.679189831018448e-05,"max_abs_dlogit":0.0055828094482421875,"author_head_fp32_same_hidden":{"max_abs_dp":5.519390106201172e-05,"mean_abs_dp":3.679081176718076e-05},"vs_author_head_same_hidden_max_abs_dp":2.7939677238464355e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_e1d610bd14e3d16c09b9","arm":"text","variant":"base","tokens":181,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2790830039884895,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.337860107421875e-05,"mean_abs_dp":2.228282392024994e-05,"max_abs_dlogit":0.0020165443420410156,"author_head_fp32_same_hidden":{"max_abs_dp":3.337860107421875e-05,"mean_abs_dp":2.2277856866518658e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_22ec57822db50ec121cf","arm":"text","variant":"base","tokens":171,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26333300047554076,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.0371208190917969e-05,"mean_abs_dp":6.886509557565053e-06,"max_abs_dlogit":0.0014500617980957031,"author_head_fp32_same_hidden":{"max_abs_dp":1.0371208190917969e-05,"mean_abs_dp":6.886509557565053e-06},"vs_author_head_same_hidden_max_abs_dp":1.0244548320770264e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_3e75f2b84e8e08e3d3f2","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2550839853938669,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00022258982062339783,"mean_abs_dp":0.00014840935667355856,"max_abs_dlogit":0.007502317428588867,"author_head_fp32_same_hidden":{"max_abs_dp":0.00022262707352638245,"mean_abs_dp":0.00014839693903923035},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_ebd74aa05324a1556288","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2504159929230809,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0001131892204284668,"mean_abs_dp":7.550356288750966e-05,"max_abs_dlogit":0.0033845901489257812,"author_head_fp32_same_hidden":{"max_abs_dp":0.00011330842971801758,"mean_abs_dp":7.554702460765839e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_d7e84e4245a29817da1f","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2566250041127205,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.277727127075195e-05,"mean_abs_dp":4.85380490620931e-05,"max_abs_dlogit":0.002319812774658203,"author_head_fp32_same_hidden":{"max_abs_dp":7.277727127075195e-05,"mean_abs_dp":4.853649685780207e-05},"vs_author_head_same_hidden_max_abs_dp":4.6566128730773926e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_f41ae66b3c09630c7a42","arm":"text","variant":"base","tokens":178,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2795000036712736,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00012940168380737305,"mean_abs_dp":8.626437435547511e-05,"max_abs_dlogit":0.004309654235839844,"author_head_fp32_same_hidden":{"max_abs_dp":0.00012940168380737305,"mean_abs_dp":8.626934140920639e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_82c2cd97c160ad27951b","arm":"text","variant":"base","tokens":171,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2886249858420342,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0002600476145744324,"mean_abs_dp":0.00017338618636131287,"max_abs_dlogit":0.004979133605957031,"author_head_fp32_same_hidden":{"max_abs_dp":0.0002600476145744324,"mean_abs_dp":0.00017339487870534262},"vs_author_head_same_hidden_max_abs_dp":2.60770320892334e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_7ada97b6f99f4788e5c7","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25991699658334255,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.5703182220458984e-05,"mean_abs_dp":2.3816712200641632e-05,"max_abs_dlogit":0.00393223762512207,"author_head_fp32_same_hidden":{"max_abs_dp":3.5822391510009766e-05,"mean_abs_dp":2.3858932157357533e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4d7a003342db5b0a17e3","arm":"text","variant":"base","tokens":187,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2775000175461173,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.696317672729492e-05,"mean_abs_dp":5.79847643772761e-05,"max_abs_dlogit":0.0032820701599121094,"author_head_fp32_same_hidden":{"max_abs_dp":8.696317672729492e-05,"mean_abs_dp":5.800028642018636e-05},"vs_author_head_same_hidden_max_abs_dp":3.166496753692627e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_19d3d8d0bf773e88be7b","arm":"text","variant":"base","tokens":172,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25125002139247954,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.5139579772949219e-05,"mean_abs_dp":1.0049901902675629e-05,"max_abs_dlogit":0.0005817413330078125,"author_head_fp32_same_hidden":{"max_abs_dp":1.5020370483398438e-05,"mean_abs_dp":1.0006129741668701e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_d6de731cca52359b1c87","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2549579949118197,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.003719151020050049,"mean_abs_dp":0.002479424079259237,"max_abs_dlogit":0.01807570457458496,"author_head_fp32_same_hidden":{"max_abs_dp":0.0037192106246948242,"mean_abs_dp":0.0024794762333234153},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b45c63fb92729af27f05","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28408298385329545,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0017579197883605957,"mean_abs_dp":0.001171929140885671,"max_abs_dlogit":0.012160301208496094,"author_head_fp32_same_hidden":{"max_abs_dp":0.0017579197883605957,"mean_abs_dp":0.0011719167232513428},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_46b7029b9a704138b77a","arm":"text","variant":"base","tokens":195,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.262790999840945,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000141143798828125,"mean_abs_dp":9.410517911116283e-05,"max_abs_dlogit":0.004653453826904297,"author_head_fp32_same_hidden":{"max_abs_dp":0.000141143798828125,"mean_abs_dp":9.409400324026744e-05},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_caf773b23934fa7362a1","arm":"text","variant":"base","tokens":190,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2630829985719174,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00014907121658325195,"mean_abs_dp":9.935783843199412e-05,"max_abs_dlogit":0.0043354034423828125,"author_head_fp32_same_hidden":{"max_abs_dp":0.00014907121658325195,"mean_abs_dp":9.935411314169566e-05},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_f46f392ef9e9e9df564b","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.751083018258214,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.071826934814453e-05,"mean_abs_dp":6.042793393135071e-05,"max_abs_dlogit":0.003118157386779785,"author_head_fp32_same_hidden":{"max_abs_dp":9.071826934814453e-05,"mean_abs_dp":6.0442835092544556e-05},"vs_author_head_same_hidden_max_abs_dp":2.60770320892334e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_005dcf6d2c0f21d64796","arm":"text","variant":"base","tokens":209,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4937080084346235,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00026667118072509766,"mean_abs_dp":0.0001777807871500651,"max_abs_dlogit":0.006699085235595703,"author_head_fp32_same_hidden":{"max_abs_dp":0.00026667118072509766,"mean_abs_dp":0.00017779072125752768},"vs_author_head_same_hidden_max_abs_dp":5.587935447692871e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_edb708bd0fe5fd67d6ee","arm":"text","variant":"base","tokens":210,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4693329974543303,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00035506486892700195,"mean_abs_dp":0.00023671860496203104,"max_abs_dlogit":0.003998517990112305,"author_head_fp32_same_hidden":{"max_abs_dp":0.00035506486892700195,"mean_abs_dp":0.00023672481377919516},"vs_author_head_same_hidden_max_abs_dp":1.862645149230957e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_97a35499bd8717f3e786","arm":"text","variant":"base","tokens":203,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2916670055128634,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0001862049102783203,"mean_abs_dp":0.00012409687042236328,"max_abs_dlogit":0.0061626434326171875,"author_head_fp32_same_hidden":{"max_abs_dp":0.0001862049102783203,"mean_abs_dp":0.0001241037001212438},"vs_author_head_same_hidden_max_abs_dp":3.3527612686157227e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_31eb4533d2c89959ec46","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2947080065496266,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00019150972366333008,"mean_abs_dp":0.0001277091602484385,"max_abs_dlogit":0.006827354431152344,"author_head_fp32_same_hidden":{"max_abs_dp":0.00019149109721183777,"mean_abs_dp":0.00012766942381858826},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_0aa24a6701ac9c56c4a0","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2786249970085919,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004139542579650879,"mean_abs_dp":0.0002759825438261032,"max_abs_dlogit":0.00974583625793457,"author_head_fp32_same_hidden":{"max_abs_dp":0.0004139542579650879,"mean_abs_dp":0.00027597323060035706},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_796d5c0da6eaadae6a3f","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2824580005835742,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.821487426757812e-06,"mean_abs_dp":5.854604144891103e-06,"max_abs_dlogit":0.0005381107330322266,"author_head_fp32_same_hidden":{"max_abs_dp":8.821487426757812e-06,"mean_abs_dp":5.8514997363090515e-06},"vs_author_head_same_hidden_max_abs_dp":4.6566128730773926e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_599b3092b6fd18312d23","arm":"text","variant":"base","tokens":187,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2746660029515624,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0003300309181213379,"mean_abs_dp":0.00021999267240365347,"max_abs_dlogit":0.005089402198791504,"author_head_fp32_same_hidden":{"max_abs_dp":0.0003299117088317871,"mean_abs_dp":0.00021993989745775858},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_960a4bcd27363e6db4b9","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27629200485534966,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0010715723037719727,"mean_abs_dp":0.0007143914699554443,"max_abs_dlogit":0.022474288940429688,"author_head_fp32_same_hidden":{"max_abs_dp":0.0010714977979660034,"mean_abs_dp":0.0007143567005793253},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_c6a411e7f2bbdf377173","arm":"text","variant":"base","tokens":207,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2946249733213335,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00021141767501831055,"mean_abs_dp":0.00014090662201245627,"max_abs_dlogit":0.005940675735473633,"author_head_fp32_same_hidden":{"max_abs_dp":0.00021135807037353516,"mean_abs_dp":0.0001408730943997701},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_e332aa927d0c2ed044aa","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2772499865386635,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.9166618585586548e-06,"mean_abs_dp":1.2485931317011516e-06,"max_abs_dlogit":0.0003085136413574219,"author_head_fp32_same_hidden":{"max_abs_dp":1.9390136003494263e-06,"mean_abs_dp":1.2479722499847412e-06},"vs_author_head_same_hidden_max_abs_dp":2.421438694000244e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_a0128e551d095f3449f8","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2820410009007901,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00013750791549682617,"mean_abs_dp":9.167380630970001e-05,"max_abs_dlogit":0.003545045852661133,"author_head_fp32_same_hidden":{"max_abs_dp":0.00013750791549682617,"mean_abs_dp":9.167380630970001e-05},"vs_author_head_same_hidden_max_abs_dp":0.0,"argmax_equal_author_head_same_hidden":true},{"id":"semif_46d6a0731bd288c370f0","arm":"text","variant":"base","tokens":182,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2550839853938669,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00014454126358032227,"mean_abs_dp":9.636208415031433e-05,"max_abs_dlogit":0.0031507015228271484,"author_head_fp32_same_hidden":{"max_abs_dp":0.00014454126358032227,"mean_abs_dp":9.634718298912048e-05},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_21ca7a8b970e36b8e20f","arm":"text","variant":"base","tokens":214,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.4124999977648258,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00819706916809082,"mean_abs_dp":0.005464700361092885,"max_abs_dlogit":0.041455745697021484,"author_head_fp32_same_hidden":{"max_abs_dp":0.00819694995880127,"mean_abs_dp":0.0054646556576093035},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4aa83af95bb69e90bc2d","arm":"text","variant":"base","tokens":212,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3362499992363155,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000591576099395752,"mean_abs_dp":0.00039437909921010333,"max_abs_dlogit":0.005152225494384766,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005916357040405273,"mean_abs_dp":0.00039441386858622235},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_cdfa9eed51fafcf09b72","arm":"text","variant":"base","tokens":197,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3065000055357814,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0017167329788208008,"mean_abs_dp":0.001144486169020335,"max_abs_dlogit":0.012655258178710938,"author_head_fp32_same_hidden":{"max_abs_dp":0.0017167329788208008,"mean_abs_dp":0.0011444799602031708},"vs_author_head_same_hidden_max_abs_dp":4.470348358154297e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_d51b05bfb9d6d7f09331","arm":"text","variant":"base","tokens":203,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30637497548013926,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.007256358861923218,"mean_abs_dp":0.004837561398744583,"max_abs_dlogit":0.01585555076599121,"author_head_fp32_same_hidden":{"max_abs_dp":0.007256358861923218,"mean_abs_dp":0.004837545255819957},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_fdd902cc6e65fad7cb4f","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28545799432322383,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0018469691276550293,"mean_abs_dp":0.0012313500046730042,"max_abs_dlogit":0.024901628494262695,"author_head_fp32_same_hidden":{"max_abs_dp":0.0018471479415893555,"mean_abs_dp":0.001231426994005839},"vs_author_head_same_hidden_max_abs_dp":1.7881393432617188e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_42384af9ae56beb37ff5","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27854100335389376,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000572502613067627,"mean_abs_dp":0.00038165133446455,"max_abs_dlogit":0.012777328491210938,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005723834037780762,"mean_abs_dp":0.0003816032161315282},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_cdf7f428857e1b841cb3","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27787499129772186,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.4424324035644531e-05,"mean_abs_dp":9.599141776561737e-06,"max_abs_dlogit":0.0011893510818481445,"author_head_fp32_same_hidden":{"max_abs_dp":1.4424324035644531e-05,"mean_abs_dp":9.60410883029302e-06},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_774f7f9245f83c796d44","arm":"text","variant":"base","tokens":180,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2512499922886491,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0002422332763671875,"mean_abs_dp":0.0001614931970834732,"max_abs_dlogit":0.006153106689453125,"author_head_fp32_same_hidden":{"max_abs_dp":0.00024235248565673828,"mean_abs_dp":0.00016154473026593527},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_92ec2ffa3dd46ada1335","arm":"text","variant":"base","tokens":204,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2828330034390092,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004864335060119629,"mean_abs_dp":0.0003243274986743927,"max_abs_dlogit":0.0034973621368408203,"author_head_fp32_same_hidden":{"max_abs_dp":0.0004864335060119629,"mean_abs_dp":0.0003243200480937958},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_6149a17bc154f9c5b4a0","arm":"text","variant":"base","tokens":200,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27766599669121206,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005390644073486328,"mean_abs_dp":0.00035939862330754596,"max_abs_dlogit":0.013210773468017578,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005390644073486328,"mean_abs_dp":0.0003593812386194865},"vs_author_head_same_hidden_max_abs_dp":4.0978193283081055e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_be866e1dbec5d42d9423","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2777920162770897,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.29425048828125e-05,"mean_abs_dp":4.196291168530782e-05,"max_abs_dlogit":0.00406801700592041,"author_head_fp32_same_hidden":{"max_abs_dp":6.29425048828125e-05,"mean_abs_dp":4.1972845792770386e-05},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_97d2ed2591fc7638806c","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2827499993145466,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0065749287605285645,"mean_abs_dp":0.004383298257986705,"max_abs_dlogit":0.02728724479675293,"author_head_fp32_same_hidden":{"max_abs_dp":0.006574809551239014,"mean_abs_dp":0.004383232444524765},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4533dd91267b4d815690","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2770000137388706,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.94529914855957e-05,"mean_abs_dp":5.295065542062124e-05,"max_abs_dlogit":0.003883838653564453,"author_head_fp32_same_hidden":{"max_abs_dp":7.94529914855957e-05,"mean_abs_dp":5.294817189375559e-05},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_d880e65fef5e2993fdd0","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2686249790713191,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0023258328437805176,"mean_abs_dp":0.001550545295079549,"max_abs_dlogit":0.02131962776184082,"author_head_fp32_same_hidden":{"max_abs_dp":0.0023258328437805176,"mean_abs_dp":0.0015505552291870117},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_90494c76078ffc3c717a","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.281208980595693,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00011616945266723633,"mean_abs_dp":7.748045027256012e-05,"max_abs_dlogit":0.009435534477233887,"author_head_fp32_same_hidden":{"max_abs_dp":0.00011616945266723633,"mean_abs_dp":7.749659319718678e-05},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_9ac5cc82c4126c91174a","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2822499955072999,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.571676254272461e-05,"mean_abs_dp":3.0496468146642048e-05,"max_abs_dlogit":0.0024480819702148438,"author_head_fp32_same_hidden":{"max_abs_dp":4.5726075768470764e-05,"mean_abs_dp":3.047349552313487e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_386e10b1c1ce906473a4","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25549999554641545,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.99758529663086e-05,"mean_abs_dp":4.6675714353720345e-05,"max_abs_dlogit":0.001918792724609375,"author_head_fp32_same_hidden":{"max_abs_dp":6.99758529663086e-05,"mean_abs_dp":4.6675714353720345e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4da6aedc1139224eeff2","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27520800358615816,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.824731826782227e-05,"mean_abs_dp":4.5536396404107414e-05,"max_abs_dlogit":0.0038660764694213867,"author_head_fp32_same_hidden":{"max_abs_dp":6.824731826782227e-05,"mean_abs_dp":4.553794860839844e-05},"vs_author_head_same_hidden_max_abs_dp":4.6566128730773926e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_a22dab8ba073ff6cdb8d","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28241699328646064,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.1317973732948303e-06,"mean_abs_dp":1.4603137969970703e-06,"max_abs_dlogit":0.003208160400390625,"author_head_fp32_same_hidden":{"max_abs_dp":2.1336600184440613e-06,"mean_abs_dp":1.4230608940124512e-06},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_42c54fa2ad45743e24d8","arm":"text","variant":"base","tokens":188,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2672090195119381,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.7621164917945862e-05,"mean_abs_dp":1.8410074214140575e-05,"max_abs_dlogit":0.0013170242309570312,"author_head_fp32_same_hidden":{"max_abs_dp":2.7613714337348938e-05,"mean_abs_dp":1.8411936859289806e-05},"vs_author_head_same_hidden_max_abs_dp":1.30385160446167e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_f3d13cf184f916677def","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28070801636204123,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00034171342849731445,"mean_abs_dp":0.00022781267762184143,"max_abs_dlogit":0.0033316612243652344,"author_head_fp32_same_hidden":{"max_abs_dp":0.00034159421920776367,"mean_abs_dp":0.00022775928179423013},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_1105577da4c8dad4609d","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2675409778021276,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.010464847087860107,"mean_abs_dp":0.0069765374064445496,"max_abs_dlogit":0.05074131488800049,"author_head_fp32_same_hidden":{"max_abs_dp":0.010464847087860107,"mean_abs_dp":0.006976552307605743},"vs_author_head_same_hidden_max_abs_dp":4.470348358154297e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_0ebd299739302d8b057b","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27699998463504016,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.001270301640033722,"mean_abs_dp":0.0008468752106030782,"max_abs_dlogit":0.006929755210876465,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012703165411949158,"mean_abs_dp":0.0008468603094418844},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_eafc22c8c40df3932a8e","arm":"text","variant":"base","tokens":189,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.7690409838687629,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0013892054557800293,"mean_abs_dp":0.0009261531134446462,"max_abs_dlogit":0.01006317138671875,"author_head_fp32_same_hidden":{"max_abs_dp":0.0013892054557800293,"mean_abs_dp":0.000926140695810318},"vs_author_head_same_hidden_max_abs_dp":3.725290298461914e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_0aa1447680e4f25862bb","arm":"text","variant":"base","tokens":208,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.48149999929592013,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.007522463798522949,"mean_abs_dp":0.005014980832735698,"max_abs_dlogit":0.06349658966064453,"author_head_fp32_same_hidden":{"max_abs_dp":0.007522463798522949,"mean_abs_dp":0.005014963448047638},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_980e76145c925747220c","arm":"text","variant":"base","tokens":209,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.47599998652003706,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0009320229291915894,"mean_abs_dp":0.0006213237841924032,"max_abs_dlogit":0.00622248649597168,"author_head_fp32_same_hidden":{"max_abs_dp":0.0009321272373199463,"mean_abs_dp":0.0006213958064715067},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4aee37a6a155747766aa","arm":"text","variant":"base","tokens":206,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.32862499938346446,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00016642361879348755,"mean_abs_dp":0.00011095901330312093,"max_abs_dlogit":0.004825592041015625,"author_head_fp32_same_hidden":{"max_abs_dp":0.00016640499234199524,"mean_abs_dp":0.00011095280448595683},"vs_author_head_same_hidden_max_abs_dp":1.862645149230957e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_6db17cc22bd656d78558","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3113339771516621,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00018066167831420898,"mean_abs_dp":0.00012047030031681061,"max_abs_dlogit":0.0048635005950927734,"author_head_fp32_same_hidden":{"max_abs_dp":0.00018066167831420898,"mean_abs_dp":0.0001204634706179301},"vs_author_head_same_hidden_max_abs_dp":1.30385160446167e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_ef3e0f8e657bf59f8a29","arm":"text","variant":"base","tokens":200,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.267458992311731,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.007280170917510986,"mean_abs_dp":0.004853422443072001,"max_abs_dlogit":0.0876457691192627,"author_head_fp32_same_hidden":{"max_abs_dp":0.0072800517082214355,"mean_abs_dp":0.004853352904319763},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_f756e1cb04e11766341e","arm":"text","variant":"base","tokens":206,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27416698867455125,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.010642945766448975,"mean_abs_dp":0.0070952574412028,"max_abs_dlogit":0.0380781888961792,"author_head_fp32_same_hidden":{"max_abs_dp":0.010642826557159424,"mean_abs_dp":0.0070951879024505615},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_d167e51bb1f499099645","arm":"text","variant":"base","tokens":186,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.3169169940520078,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.004296362400054932,"mean_abs_dp":0.0028642316659291587,"max_abs_dlogit":0.04268360137939453,"author_head_fp32_same_hidden":{"max_abs_dp":0.004296302795410156,"mean_abs_dp":0.0028642117977142334},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_674ca4e9346216255466","arm":"text","variant":"base","tokens":176,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27287498232908547,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00017136335372924805,"mean_abs_dp":0.00011424471934636433,"max_abs_dlogit":0.004530668258666992,"author_head_fp32_same_hidden":{"max_abs_dp":0.00017136335372924805,"mean_abs_dp":0.00011424906551837921},"vs_author_head_same_hidden_max_abs_dp":1.30385160446167e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_1df851a202994a07f123","arm":"text","variant":"base","tokens":194,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2815420157276094,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00012897327542304993,"mean_abs_dp":8.599087595939636e-05,"max_abs_dlogit":0.004180908203125,"author_head_fp32_same_hidden":{"max_abs_dp":0.00012900307774543762,"mean_abs_dp":8.599335948626201e-05},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b5e6788d95b8395d2888","arm":"text","variant":"base","tokens":193,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27620900073088706,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.2479248046875e-05,"mean_abs_dp":4.8308943708737694e-05,"max_abs_dlogit":0.00505298376083374,"author_head_fp32_same_hidden":{"max_abs_dp":7.2479248046875e-05,"mean_abs_dp":4.830832282702128e-05},"vs_author_head_same_hidden_max_abs_dp":1.862645149230957e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_32f36cd0fbbfb3604586","arm":"text","variant":"base","tokens":183,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2611250092741102,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.9596110582351685e-05,"mean_abs_dp":2.6363258560498554e-05,"max_abs_dlogit":0.0043926239013671875,"author_head_fp32_same_hidden":{"max_abs_dp":3.961101174354553e-05,"mean_abs_dp":2.640858292579651e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_843185706adff0c60f8b","arm":"text","variant":"base","tokens":184,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.25383298634551466,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0011990368366241455,"mean_abs_dp":0.0007993355393409729,"max_abs_dlogit":0.01717829704284668,"author_head_fp32_same_hidden":{"max_abs_dp":0.001198992133140564,"mean_abs_dp":0.0007993293305238088},"vs_author_head_same_hidden_max_abs_dp":4.470348358154297e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b531dd603bf080dab820","arm":"text","variant":"base","tokens":199,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2802920062094927,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00018590688705444336,"mean_abs_dp":0.00012391060590744019,"max_abs_dlogit":0.008097410202026367,"author_head_fp32_same_hidden":{"max_abs_dp":0.00018590688705444336,"mean_abs_dp":0.00012390377620855966},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_be2dd67bc733c1213bef","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.28037498123012483,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00030612945556640625,"mean_abs_dp":0.00020407761136690775,"max_abs_dlogit":0.009941339492797852,"author_head_fp32_same_hidden":{"max_abs_dp":0.00030612945556640625,"mean_abs_dp":0.00020409623781840006},"vs_author_head_same_hidden_max_abs_dp":3.3527612686157227e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b8e2750307df9761e716","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.27904199669137597,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00014644861221313477,"mean_abs_dp":9.761626521746318e-05,"max_abs_dlogit":0.006569981575012207,"author_head_fp32_same_hidden":{"max_abs_dp":0.00014656782150268555,"mean_abs_dp":9.766283134619395e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_5d31b4bf2d0fc193f53b","arm":"text","variant":"base","tokens":201,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.29987498419359326,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0003349184989929199,"mean_abs_dp":0.00022328210373719534,"max_abs_dlogit":0.007233858108520508,"author_head_fp32_same_hidden":{"max_abs_dp":0.0003349184989929199,"mean_abs_dp":0.0002232914169629415},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_a9230edfc6ec9e524af0","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2638750011101365,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":7.283687591552734e-05,"mean_abs_dp":4.85380490620931e-05,"max_abs_dlogit":0.002293109893798828,"author_head_fp32_same_hidden":{"max_abs_dp":7.283687591552734e-05,"mean_abs_dp":4.853494465351105e-05},"vs_author_head_same_hidden_max_abs_dp":9.313225746154785e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_c100e84585149a04be49","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2575840044301003,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":3.606081008911133e-05,"mean_abs_dp":2.40374356508255e-05,"max_abs_dlogit":0.0020672082901000977,"author_head_fp32_same_hidden":{"max_abs_dp":3.606081008911133e-05,"mean_abs_dp":2.4037125209967296e-05},"vs_author_head_same_hidden_max_abs_dp":6.51925802230835e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_579beed184ffc1104aa9","arm":"text","variant":"base","tokens":182,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.24458399275317788,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.00543212890625e-05,"mean_abs_dp":2.6752861837546032e-05,"max_abs_dlogit":0.0024967193603515625,"author_head_fp32_same_hidden":{"max_abs_dp":4.00543212890625e-05,"mean_abs_dp":2.6755034923553467e-05},"vs_author_head_same_hidden_max_abs_dp":5.587935447692871e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_c4e9bd463b6fde5be743","arm":"text","variant":"base","tokens":170,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.24358299560844898,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":8.058547973632812e-05,"mean_abs_dp":5.371216684579849e-05,"max_abs_dlogit":0.0025463104248046875,"author_head_fp32_same_hidden":{"max_abs_dp":8.058547973632812e-05,"mean_abs_dp":5.370906243721644e-05},"vs_author_head_same_hidden_max_abs_dp":9.313225746154785e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b863998d4be2721a5dfd","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2860829990822822,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00035303831100463867,"mean_abs_dp":0.00023537129163742065,"max_abs_dlogit":0.007825851440429688,"author_head_fp32_same_hidden":{"max_abs_dp":0.0003529787063598633,"mean_abs_dp":0.00023532162110010782},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_37cebc3792e98049439f","arm":"text","variant":"base","tokens":202,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.283041998045519,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00024712085723876953,"mean_abs_dp":0.00016476276020208994,"max_abs_dlogit":0.00909435749053955,"author_head_fp32_same_hidden":{"max_abs_dp":0.00024712085723876953,"mean_abs_dp":0.00016476338108380636},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_64d63f86e328aff1481c","arm":"text","variant":"base","tokens":210,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.41629199404269457,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":9.417533874511719e-06,"mean_abs_dp":6.287048260370891e-06,"max_abs_dlogit":0.0020966529846191406,"author_head_fp32_same_hidden":{"max_abs_dp":9.5367431640625e-06,"mean_abs_dp":6.327405571937561e-06},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_6d64a8459f64eac7f4cc","arm":"text","variant":"base","tokens":207,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2972499933093786,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":6.133690476417542e-05,"mean_abs_dp":4.0868918100992836e-05,"max_abs_dlogit":0.0019292831420898438,"author_head_fp32_same_hidden":{"max_abs_dp":6.134435534477234e-05,"mean_abs_dp":4.087140162785848e-05},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_2708193212d8a4a523c7","arm":"text","variant":"base","tokens":207,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.30945902108214796,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00259321928024292,"mean_abs_dp":0.0017287979523340862,"max_abs_dlogit":0.008353471755981445,"author_head_fp32_same_hidden":{"max_abs_dp":0.0025933384895324707,"mean_abs_dp":0.0017288873593012493},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_5a37d1dfcad20810a76a","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2824580005835742,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0011305809020996094,"mean_abs_dp":0.0007536895573139191,"max_abs_dlogit":0.011626720428466797,"author_head_fp32_same_hidden":{"max_abs_dp":0.0011305809020996094,"mean_abs_dp":0.0007536858320236206},"vs_author_head_same_hidden_max_abs_dp":1.1175870895385742e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_acacd8b46acd25cb954b","arm":"text","variant":"base","tokens":206,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2715839946176857,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":2.2590160369873047e-05,"mean_abs_dp":1.5050172805786133e-05,"max_abs_dlogit":0.0008940696716308594,"author_head_fp32_same_hidden":{"max_abs_dp":2.2530555725097656e-05,"mean_abs_dp":1.502657930056254e-05},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_4089c199b8b796ec3ed8","arm":"text","variant":"base","tokens":205,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2700829936657101,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.002208597958087921,"mean_abs_dp":0.0014723911881446838,"max_abs_dlogit":0.02852797508239746,"author_head_fp32_same_hidden":{"max_abs_dp":0.002208612859249115,"mean_abs_dp":0.0014723738034566243},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_02f2b95a5c5a4777a45c","arm":"text","variant":"base","tokens":191,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2433749905321747,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":1.6693957149982452e-05,"mean_abs_dp":1.1131477852662405e-05,"max_abs_dlogit":0.0022325515747070312,"author_head_fp32_same_hidden":{"max_abs_dp":1.6673468053340912e-05,"mean_abs_dp":1.1084601283073425e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"semif_ebea4681e3dcc7d92ca2","arm":"text","variant":"base","tokens":196,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2735840098466724,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0014224648475646973,"mean_abs_dp":0.0009483347336451212,"max_abs_dlogit":0.007730245590209961,"author_head_fp32_same_hidden":{"max_abs_dp":0.0014224648475646973,"mean_abs_dp":0.0009483247995376587},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"semif_b8c5b9b285cbdddc9f8d","arm":"text","variant":"base","tokens":187,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.26195801910944283,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":4.363059997558594e-05,"mean_abs_dp":2.908427268266678e-05,"max_abs_dlogit":0.002480030059814453,"author_head_fp32_same_hidden":{"max_abs_dp":4.363059997558594e-05,"mean_abs_dp":2.9086445768674214e-05},"vs_author_head_same_hidden_max_abs_dp":6.51925802230835e-09,"argmax_equal_author_head_same_hidden":true},{"id":"semif_7ed75d854d99b3a81916","arm":"text","variant":"base","tokens":185,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":3,"layout":[[0,3]],"host_ms":0.2531660138629377,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"semif_authored144","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0023435354232788086,"mean_abs_dp":0.0015623581906159718,"max_abs_dlogit":0.020920276641845703,"author_head_fp32_same_hidden":{"max_abs_dp":0.0023436546325683594,"mean_abs_dp":0.0015624519437551498},"vs_author_head_same_hidden_max_abs_dp":1.4156103134155273e-07,"argmax_equal_author_head_same_hidden":true}],"readout":{"path":"<lane>/results/readout_fp16_pf64_full.json","sha256":"89e16d622d644dd6293b0a443991d819bce8128a0fc464b38d5cbad8040c97e4","bundle":"clef_flash_decode_fp16_pf64","result":"PASS","author_head_summary":{"max_abs_dp":0.011243879795074463,"mean_of_run_mean_abs_dp":0.0005416686907504593,"argmax_equal_non_near_tie":397,"questions_non_near_tie":397,"argmax_equal_near_tie":4,"near_tie_questions":4},"vs_author_head_same_hidden":{"max_abs_dp":1.7881393432617188e-07,"argmax_equal_runs":213}}},"h2_heldout":{"schema":"clef-flash-head-h2/1","gate":"end to end in Python: the decoder bundle's fp16 hidden (readout shards, not re-run) -> host span matrices + fp16-table lexical -> head graph -> per-question fp32 softmax","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02},"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite_and_padding_zero":true,"rerun_bit_equal":true},"result":"PASS","summary":{"runs":40,"questions":188,"argmax_equal":188,"questions_non_near_tie":186,"argmax_equal_non_near_tie":186,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.008473336696624756,"mean_of_run_mean_abs_dp":0.0003611294226269868,"max_abs_dlogit":0.05001688003540039,"worst_run":"ho_i06/g448"},"by_source_arm":{"heldout_image/g256":{"runs":8,"questions":31,"argmax_equal":31,"questions_non_near_tie":31,"argmax_equal_non_near_tie":31,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0019000321626663208,"mean_of_run_mean_abs_dp":0.00024638341676916126,"max_abs_dlogit":0.021762490272521973,"worst_run":"ho_i05/g256"},"heldout_image/g448":{"runs":8,"questions":31,"argmax_equal":31,"questions_non_near_tie":31,"argmax_equal_non_near_tie":31,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.008473336696624756,"mean_of_run_mean_abs_dp":0.0002947909696188637,"max_abs_dlogit":0.044218629598617554,"worst_run":"ho_i06/g448"},"heldout_json/text":{"runs":8,"questions":36,"argmax_equal":36,"questions_non_near_tie":35,"argmax_equal_non_near_tie":35,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.008229672908782959,"mean_of_run_mean_abs_dp":0.0005812550225172544,"max_abs_dlogit":0.05001688003540039,"worst_run":"ho_j06/text"},"heldout_photo/g256":{"runs":2,"questions":10,"argmax_equal":10,"questions_non_near_tie":10,"argmax_equal_non_near_tie":10,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0019177794456481934,"mean_of_run_mean_abs_dp":0.0002579948253126121,"max_abs_dlogit":0.049265652894973755,"worst_run":"ho_p02/g256"},"heldout_photo/g448":{"runs":2,"questions":10,"argmax_equal":10,"questions_non_near_tie":10,"argmax_equal_non_near_tie":10,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0004938840866088867,"mean_of_run_mean_abs_dp":0.0001009312289420004,"max_abs_dlogit":0.01087874174118042,"worst_run":"ho_p02/g448"},"heldout_text/text":{"runs":10,"questions":56,"argmax_equal":56,"questions_non_near_tie":55,"argmax_equal_non_near_tie":55,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.006222933530807495,"mean_of_run_mean_abs_dp":0.00043936175704711985,"max_abs_dlogit":0.03148841857910156,"worst_run":"ho_t07/text"},"heldout_text_long/text":{"runs":2,"questions":14,"argmax_equal":14,"questions_non_near_tie":14,"argmax_equal_non_near_tie":14,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0012440681457519531,"mean_of_run_mean_abs_dp":0.00017713597742840647,"max_abs_dlogit":0.0157163143157959,"worst_run":"ho_l01/text"}},"head_bundle":{"dir":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","metadata_sha256":"631219aafcdb56ef1f008e5476f0e7a282933abdc51b4835132676d97124ab20","aimodelc":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_metadata":{"graph":"conversion/clef_flash/clef_head.py ClefHeadGraph: the author's JointSchemaHead for one record, tensor inputs only","shape":"bucket","functions":["t512","t1024","t2048","t4096"],"buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"dims":null,"host_pad":"bucket","host_pad_rule":"T to the smallest bucket >= T (512, 1024, 2048, 4096), Q to 16, O to 128; padding rows / columns zero, their valid flags 0","inputs":{"hidden":"[T, 4096] f32","key_valid":"[T] f32","q_avg":"[Q, T] f32","o_avg":"[O, T] f32","g_avg":"[1, T] f32","lexical":"[O, 4096] f32","member":"[Q, O] f32","type_ids":"[Q] i32","q_valid":"[Q] f32","o_valid":"[O] f32"},"input_order":["hidden","key_valid","q_avg","o_avg","g_avg","lexical","member","type_ids","q_valid","o_valid"],"outputs":{"logits":"[O] f32; padding options 0"},"weights":"fp16w32","softmax":"host: per question, fp32, over that question's options","lexical":"host: lm_head_fp16.bin rows of each option span's ids, averaged (clef_head.lexical_rows)","source":{"hf_id":"Cloudflare/clef-flash","revision":"17f0b0ad64efb65d273590632833508766b2aae6","joint_schema_model_sha256":"0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3","joint_head_safetensors_sha256":"19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0","joint_head_config_sha256":"77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0"}}},"descriptors":{"t512":{"function":"t512","inputs":{"hidden":[[512,4096],"float32"],"key_valid":[[512],"float32"],"q_avg":[[16,512],"float32"],"o_avg":[[128,512],"float32"],"g_avg":[[1,512],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t1024":{"function":"t1024","inputs":{"hidden":[[1024,4096],"float32"],"key_valid":[[1024],"float32"],"q_avg":[[16,1024],"float32"],"o_avg":[[128,1024],"float32"],"g_avg":[[1,1024],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t2048":{"function":"t2048","inputs":{"hidden":[[2048,4096],"float32"],"key_valid":[[2048],"float32"],"q_avg":[[16,2048],"float32"],"o_avg":[[128,2048],"float32"],"g_avg":[[1,2048],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t4096":{"function":"t4096","inputs":{"hidden":[[4096,4096],"float32"],"key_valid":[[4096],"float32"],"q_avg":[[16,4096],"float32"],"o_avg":[[128,4096],"float32"],"g_avg":[[1,4096],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}}},"descriptors_same_every_process":true,"oracle":{"path":"<lane>/oracle/heldout/records_oracle.json","sha256":"85e4dda56890dff47a163bdd74c85291f5d3be0944403dd1f59ba2f5b9faaa3f","runs":40},"table":{"path":"<lane>/exports/host/lm_head_fp16.bin","json":"<lane>/exports/host/lm_head_fp16.json"},"near_ties":[{"id":"ho_t07","arm":"text","question_id":"questions_asked","oracle_top2_margin":0.011684805154800415,"argmax_equal":true,"max_abs_dp":0.006222933530807495,"probs":[0.009835842996835709,0.4039638340473175,0.40109479427337646,0.18510542809963226],"probs_oracle":[0.009856987744569778,0.410186767578125,0.3985019624233246,0.18145422637462616]},{"id":"ho_j01","arm":"text","question_id":"late_start","oracle_top2_margin":0.0010575354099273682,"argmax_equal":true,"max_abs_dp":0.00018167495727539062,"probs":[0.5003471374511719,0.4996528923511505],"probs_oracle":[0.5005287528038025,0.4994712173938751]}],"timing":{"contended":true,"by_T":{"300<T<=700":{"runs":16,"median_tokens":595.5,"median_padded_T":1024.0,"first_call_ms_median":8.275708503788337,"second_call_ms_median":7.8743539925199,"host_ms_median":2.452520508086309},"700<T<=1500":{"runs":22,"median_tokens":767.5,"median_padded_T":1024.0,"first_call_ms_median":8.32558301044628,"second_call_ms_median":7.8693129908060655,"host_ms_median":3.9027705061016604},"T>1500":{"runs":2,"median_tokens":2790.5,"median_padded_T":4096.0,"first_call_ms_median":20.457228980376385,"second_call_ms_median":19.1939375072252,"host_ms_median":9.602770995115861}},"all_first_call_ms_median":8.32558301044628,"all_second_call_ms_median":7.881354496930726},"processes":[{"pid":18813,"started":1791004714.5753028,"load_model_seconds":0.006053499993868172,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.32335299998521805,"t1024":0.0018566249927971512,"t2048":0.0011412909952923656,"t4096":0.001106833020457998},"reset_check":{"run":["ho_t01","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791004715.91495,"process_wall_seconds":1.8270283330057282,"runs":40}],"red_arm":null,"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max"},"script":{"gate_head.py":"1d66d14945d23d15b80866799a6287a08fcca41cf95ff56ea41e418a8c0b5e0b","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d"},"seconds":{"gpu_processes":1.8293819170212373},"generated_at":"2026-10-03T05:18:36+00:00","runs":[{"id":"ho_t01","arm":"text","variant":"base","tokens":708,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17]],"host_ms":9.207541996147484,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0023886561393737793,"mean_abs_dp":0.00033675205400761436,"max_abs_dlogit":0.023274660110473633,"author_head_fp32_same_hidden":{"max_abs_dp":0.0023886561393737793,"mean_abs_dp":0.0003367487943786032},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t02","arm":"text","variant":"base","tokens":746,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,2],[2,7],[7,9],[9,13],[13,16]],"host_ms":9.57516601192765,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":1,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0018744468688964844,"mean_abs_dp":0.0003860886790789664,"max_abs_dlogit":0.01286935806274414,"author_head_fp32_same_hidden":{"max_abs_dp":0.001874387264251709,"mean_abs_dp":0.0003860545693896711},"vs_author_head_same_hidden_max_abs_dp":2.384185791015625e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t03","arm":"text","variant":"base","tokens":842,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":20,"layout":[[0,2],[2,6],[6,8],[8,12],[12,16],[16,20]],"host_ms":9.003416984342039,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":2,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00241011381149292,"mean_abs_dp":0.00028710811748169365,"max_abs_dlogit":0.009614109992980957,"author_head_fp32_same_hidden":{"max_abs_dp":0.0024101734161376953,"mean_abs_dp":0.00028711294289678334},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t04","arm":"text","variant":"base","tokens":658,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,5],[5,7],[7,9],[9,16]],"host_ms":4.867958021350205,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":3,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0008854269981384277,"mean_abs_dp":0.00013302513980306685,"max_abs_dlogit":0.010327696800231934,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008855462074279785,"mean_abs_dp":0.000133035471662879},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t05","arm":"text","variant":"base","tokens":727,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,6],[6,8],[8,10],[10,14]],"host_ms":7.360124989645556,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":4,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0002464801073074341,"mean_abs_dp":9.086175954767636e-05,"max_abs_dlogit":0.009530067443847656,"author_head_fp32_same_hidden":{"max_abs_dp":0.00024646520614624023,"mean_abs_dp":9.085883253387042e-05},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t06","arm":"text","variant":"base","tokens":721,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,7],[7,11],[11,13],[13,17]],"host_ms":6.71004198375158,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":5,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0009756684303283691,"mean_abs_dp":0.00017992193427156,"max_abs_dlogit":0.0075531005859375,"author_head_fp32_same_hidden":{"max_abs_dp":0.0009757280349731445,"mean_abs_dp":0.00017993541105705148},"vs_author_head_same_hidden_max_abs_dp":7.450580596923828e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t07","arm":"text","variant":"base","tokens":1086,"function":"t2048","shape":{"T":2048,"Q":16,"O":128},"options":29,"layout":[[0,5],[5,9],[9,13],[13,15],[15,17],[17,21],[21,25],[25,29]],"host_ms":4.698125005234033,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":6,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.006222933530807495,"mean_abs_dp":0.0007646634275543278,"max_abs_dlogit":0.03148841857910156,"author_head_fp32_same_hidden":{"max_abs_dp":0.006222903728485107,"mean_abs_dp":0.0007646556397707298},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t08","arm":"text","variant":"base","tokens":864,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":20,"layout":[[0,5],[5,9],[9,13],[13,15],[15,17],[17,20]],"host_ms":4.64887500857003,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":7,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.002567768096923828,"mean_abs_dp":0.00043065121863037346,"max_abs_dlogit":0.016382813453674316,"author_head_fp32_same_hidden":{"max_abs_dp":0.0025677084922790527,"mean_abs_dp":0.00043065219651907685},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t09","arm":"text","variant":"base","tokens":867,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,3],[3,5],[5,7],[7,11],[11,15],[15,18]],"host_ms":4.731749999336898,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":8,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0032619237899780273,"mean_abs_dp":0.0009531376676426993,"max_abs_dlogit":0.01867353916168213,"author_head_fp32_same_hidden":{"max_abs_dp":0.0032618045806884766,"mean_abs_dp":0.0009531206192655696},"vs_author_head_same_hidden_max_abs_dp":1.7881393432617188e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_t10","arm":"text","variant":"base","tokens":800,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":21,"layout":[[0,5],[5,7],[7,9],[9,13],[13,17],[17,21]],"host_ms":4.214457992929965,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":9,"source":"heldout_text","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0036197900772094727,"mean_abs_dp":0.0008314075724532207,"max_abs_dlogit":0.010805219411849976,"author_head_fp32_same_hidden":{"max_abs_dp":0.0036199092864990234,"mean_abs_dp":0.0008314249127925862},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j01","arm":"text","variant":"base","tokens":769,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,4],[4,8],[8,10],[10,12],[12,15]],"host_ms":2.857415995094925,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":10,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0005374550819396973,"mean_abs_dp":0.00014550617585579556,"max_abs_dlogit":0.011201620101928711,"author_head_fp32_same_hidden":{"max_abs_dp":0.0005373954772949219,"mean_abs_dp":0.0001455209528406461},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j02","arm":"text","variant":"base","tokens":335,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":6,"layout":[[0,6]],"host_ms":0.37966700620017946,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":11,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0011948645114898682,"mean_abs_dp":0.0004763146862387657,"max_abs_dlogit":0.013052940368652344,"author_head_fp32_same_hidden":{"max_abs_dp":0.0011948943138122559,"mean_abs_dp":0.0004763215159376462},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j03","arm":"text","variant":"base","tokens":775,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":19,"layout":[[0,2],[2,8],[8,12],[12,14],[14,19]],"host_ms":4.077666002558544,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":12,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0018700957298278809,"mean_abs_dp":0.0002910656070238666,"max_abs_dlogit":0.016313552856445312,"author_head_fp32_same_hidden":{"max_abs_dp":0.00186997652053833,"mean_abs_dp":0.00029106947936509783},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j04","arm":"text","variant":"base","tokens":610,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":12,"layout":[[0,2],[2,6],[6,10],[10,12]],"host_ms":3.562666999641806,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":13,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00146406888961792,"mean_abs_dp":0.00042268486383060616,"max_abs_dlogit":0.05001688003540039,"author_head_fp32_same_hidden":{"max_abs_dp":0.00146406888961792,"mean_abs_dp":0.00042267834457258385},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j05","arm":"text","variant":"base","tokens":775,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,7],[7,9],[9,13],[13,17]],"host_ms":2.5227079750038683,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":14,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0017920732498168945,"mean_abs_dp":0.00044073186376515557,"max_abs_dlogit":0.009947538375854492,"author_head_fp32_same_hidden":{"max_abs_dp":0.00179213285446167,"mean_abs_dp":0.00044074375182390213},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j06","arm":"text","variant":"base","tokens":733,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,2],[2,5],[5,7],[7,11],[11,13],[13,18]],"host_ms":3.49279199144803,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":15,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.008229672908782959,"mean_abs_dp":0.001465221271953649,"max_abs_dlogit":0.035484910011291504,"author_head_fp32_same_hidden":{"max_abs_dp":0.008229732513427734,"mean_abs_dp":0.0014652300160378218},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j07","arm":"text","variant":"base","tokens":826,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,5],[5,10],[10,14],[14,16],[16,18]],"host_ms":3.7278750096447766,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":16,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0015503168106079102,"mean_abs_dp":0.0003872389659389026,"max_abs_dlogit":0.03403615951538086,"author_head_fp32_same_hidden":{"max_abs_dp":0.0015503168106079102,"mean_abs_dp":0.000387246015533391},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_j08","arm":"text","variant":"base","tokens":798,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":18,"layout":[[0,2],[2,7],[7,11],[11,14],[14,18]],"host_ms":5.2987500093877316,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":17,"source":"heldout_json","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0043204426765441895,"mean_abs_dp":0.001021276745531294,"max_abs_dlogit":0.013165920972824097,"author_head_fp32_same_hidden":{"max_abs_dp":0.0043204426765441895,"mean_abs_dp":0.0010212610165278118},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_l01","arm":"text","variant":"base","tokens":2850,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":25,"layout":[[0,5],[5,9],[9,11],[11,15],[15,17],[17,21],[21,25]],"host_ms":12.81462499173358,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":18,"source":"heldout_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012440681457519531,"mean_abs_dp":0.0002137862518429756,"max_abs_dlogit":0.0157163143157959,"author_head_fp32_same_hidden":{"max_abs_dp":0.0012440979480743408,"mean_abs_dp":0.00021379227750003337},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_l02","arm":"text","variant":"base","tokens":2731,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":24,"layout":[[0,5],[5,9],[9,11],[11,15],[15,19],[19,21],[21,24]],"host_ms":6.390916998498142,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":19,"source":"heldout_text_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006785988807678223,"mean_abs_dp":0.00014048570301383734,"max_abs_dlogit":0.012473464012145996,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006784796714782715,"mean_abs_dp":0.00014047938748262823},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i01","arm":"g256","variant":"base","tokens":529,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":3.6533339880406857,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":20,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0018740296363830566,"mean_abs_dp":0.00034802532396637474,"max_abs_dlogit":0.008925378322601318,"author_head_fp32_same_hidden":{"max_abs_dp":0.0018739700317382812,"mean_abs_dp":0.00034801242873072624},"vs_author_head_same_hidden_max_abs_dp":8.940696716308594e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i01","arm":"g448","variant":"base","tokens":661,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,6],[6,10],[10,13]],"host_ms":0.7009159890003502,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":21,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006813406944274902,"mean_abs_dp":0.00014678890315385966,"max_abs_dlogit":0.007400035858154297,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006813406944274902,"mean_abs_dp":0.00014678926135484988},"vs_author_head_same_hidden_max_abs_dp":1.862645149230957e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i02","arm":"g256","variant":"base","tokens":570,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,8],[8,12],[12,14]],"host_ms":2.301125001395121,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":22,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0002873539924621582,"mean_abs_dp":9.929781247462546e-05,"max_abs_dlogit":0.011154472827911377,"author_head_fp32_same_hidden":{"max_abs_dp":0.0002872347831726074,"mean_abs_dp":9.92872186803392e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i02","arm":"g448","variant":"base","tokens":702,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":14,"layout":[[0,4],[4,8],[8,12],[12,14]],"host_ms":0.7359580195043236,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":23,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0007925629615783691,"mean_abs_dp":0.00017766930562044893,"max_abs_dlogit":0.013583004474639893,"author_head_fp32_same_hidden":{"max_abs_dp":0.0007925629615783691,"mean_abs_dp":0.00017766780885202543},"vs_author_head_same_hidden_max_abs_dp":1.4901161193847656e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i03","arm":"g256","variant":"base","tokens":541,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,10],[10,14],[14,16]],"host_ms":2.6039160147774965,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":24,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00014767982065677643,"mean_abs_dp":3.202553489245474e-05,"max_abs_dlogit":0.017756938934326172,"author_head_fp32_same_hidden":{"max_abs_dp":0.00014767050743103027,"mean_abs_dp":3.2029609428718686e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i03","arm":"g448","variant":"base","tokens":673,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":16,"layout":[[0,10],[10,14],[14,16]],"host_ms":0.7929159910418093,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":25,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00018352270126342773,"mean_abs_dp":3.288165316917002e-05,"max_abs_dlogit":0.003849327564239502,"author_head_fp32_same_hidden":{"max_abs_dp":0.00018352270126342773,"mean_abs_dp":3.288139123469591e-05},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i04","arm":"g256","variant":"base","tokens":437,"function":"t512","shape":{"T":512,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":1.4717500016558915,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":26,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0009416937828063965,"mean_abs_dp":0.0002738892333582044,"max_abs_dlogit":0.010123729705810547,"author_head_fp32_same_hidden":{"max_abs_dp":0.0009417235851287842,"mean_abs_dp":0.0002738974755629897},"vs_author_head_same_hidden_max_abs_dp":1.7881393432617188e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i04","arm":"g448","variant":"base","tokens":569,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":10,"layout":[[0,4],[4,6],[6,10]],"host_ms":0.6115410069469362,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":27,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.000820159912109375,"mean_abs_dp":0.00032475993502885104,"max_abs_dlogit":0.010278463363647461,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008200705051422119,"mean_abs_dp":0.0003247383749112487},"vs_author_head_same_hidden_max_abs_dp":8.940696716308594e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i05","arm":"g256","variant":"base","tokens":701,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":21,"layout":[[0,7],[7,11],[11,15],[15,17],[17,21]],"host_ms":2.1097919961903244,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":28,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0019000321626663208,"mean_abs_dp":0.00046825936685005826,"max_abs_dlogit":0.021762490272521973,"author_head_fp32_same_hidden":{"max_abs_dp":0.0019000768661499023,"mean_abs_dp":0.00046826333605817387},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i05","arm":"g448","variant":"base","tokens":833,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":21,"layout":[[0,7],[7,11],[11,15],[15,17],[17,21]],"host_ms":0.9122499905060977,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":29,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0008838772773742676,"mean_abs_dp":0.0001513165244389148,"max_abs_dlogit":0.01622319221496582,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008838772773742676,"mean_abs_dp":0.00015131175695430664},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i06","arm":"g256","variant":"base","tokens":628,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":19,"layout":[[0,10],[10,15],[15,17],[17,19]],"host_ms":0.7939590141177177,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":30,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0012934207916259766,"mean_abs_dp":0.00029432307928800583,"max_abs_dlogit":0.016123056411743164,"author_head_fp32_same_hidden":{"max_abs_dp":0.001293480396270752,"mean_abs_dp":0.0002943148199272783},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i06","arm":"g448","variant":"base","tokens":760,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":19,"layout":[[0,10],[10,15],[15,17],[17,19]],"host_ms":0.836457998957485,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":31,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.008473336696624756,"mean_abs_dp":0.0010460431344415013,"max_abs_dlogit":0.044218629598617554,"author_head_fp32_same_hidden":{"max_abs_dp":0.008473306894302368,"mean_abs_dp":0.001046040818389309},"vs_author_head_same_hidden_max_abs_dp":5.960464477539063e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i07","arm":"g256","variant":"base","tokens":535,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,7],[7,11],[11,13]],"host_ms":3.8478340138681233,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":32,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0008235126733779907,"mean_abs_dp":0.00026028134072056186,"max_abs_dlogit":0.015843629837036133,"author_head_fp32_same_hidden":{"max_abs_dp":0.0008234977722167969,"mean_abs_dp":0.0002602764333669956},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i07","arm":"g448","variant":"base","tokens":667,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":13,"layout":[[0,4],[4,7],[7,11],[11,13]],"host_ms":0.7486249960493296,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":33,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0015792250633239746,"mean_abs_dp":0.0003233253239439084,"max_abs_dlogit":0.015192508697509766,"author_head_fp32_same_hidden":{"max_abs_dp":0.0015791058540344238,"mean_abs_dp":0.00032330875714810996},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i08","arm":"g256","variant":"base","tokens":581,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,10],[10,12],[12,17]],"host_ms":2.9498749936465174,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":34,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0007407069206237793,"mean_abs_dp":0.00019496564260300468,"max_abs_dlogit":0.0068645477294921875,"author_head_fp32_same_hidden":{"max_abs_dp":0.0007407069206237793,"mean_abs_dp":0.00019496487563147266},"vs_author_head_same_hidden_max_abs_dp":2.2351741790771484e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_i08","arm":"g448","variant":"base","tokens":713,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,5],[5,10],[10,12],[12,17]],"host_ms":0.7824999920558184,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":35,"source":"heldout_image","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006317496299743652,"mean_abs_dp":0.0001555429771542549,"max_abs_dlogit":0.00944584608078003,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006317570805549622,"mean_abs_dp":0.00015553684138199862},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_p01","arm":"g256","variant":"base","tokens":634,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,2],[2,6],[6,8],[8,12],[12,15]],"host_ms":5.851416994119063,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":36,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.00023355334997177124,"mean_abs_dp":8.026597400506338e-05,"max_abs_dlogit":0.0057179927825927734,"author_head_fp32_same_hidden":{"max_abs_dp":0.00023353099822998047,"mean_abs_dp":8.026044815778732e-05},"vs_author_head_same_hidden_max_abs_dp":2.9802322387695312e-08,"argmax_equal_author_head_same_hidden":true},{"id":"ho_p01","arm":"g448","variant":"base","tokens":766,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":15,"layout":[[0,2],[2,6],[6,8],[8,12],[12,15]],"host_ms":0.8114579832181334,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":37,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0002489015460014343,"mean_abs_dp":0.00010609092811743418,"max_abs_dlogit":0.0048732757568359375,"author_head_fp32_same_hidden":{"max_abs_dp":0.0002488754689693451,"mean_abs_dp":0.00010608273247877756},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_p02","arm":"g256","variant":"base","tokens":620,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17]],"host_ms":3.6090000066906214,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":38,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0019177794456481934,"mean_abs_dp":0.00043572367662016084,"max_abs_dlogit":0.049265652894973755,"author_head_fp32_same_hidden":{"max_abs_dp":0.0019177794456481934,"mean_abs_dp":0.00043572674450628897},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true},{"id":"ho_p02","arm":"g448","variant":"base","tokens":752,"function":"t1024","shape":{"T":1024,"Q":16,"O":128},"options":17,"layout":[[0,4],[4,6],[6,10],[10,14],[14,17]],"host_ms":0.8471659966744483,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":39,"source":"heldout_photo","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0004938840866088867,"mean_abs_dp":9.577152976656661e-05,"max_abs_dlogit":0.01087874174118042,"author_head_fp32_same_hidden":{"max_abs_dp":0.0004940032958984375,"mean_abs_dp":9.578403414172285e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true}],"readout":{"path":"<lane>/results/readout_heldout_fp16_pf64.json","sha256":"49a2553cbe8ff955e47a524c1f8b8ce1f151e9998cae7ebb141a096a52c3b5e9","bundle":"clef_flash_decode_fp16_pf64","result":"PASS","author_head_summary":{"max_abs_dp":0.008473306894302368,"mean_of_run_mean_abs_dp":0.00036112793286965494,"argmax_equal_non_near_tie":186,"questions_non_near_tie":186,"argmax_equal_near_tie":2,"near_tie_questions":2},"vs_author_head_same_hidden":{"max_abs_dp":2.384185791015625e-07,"argmax_equal_runs":40}}},"h2_diag_long":{"schema":"clef-flash-head-h2/1","gate":"end to end in Python: the decoder bundle's fp16 hidden (readout shards, not re-run) -> host span matrices + fp16-table lexical -> head graph -> per-question fp32 softmax","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02},"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite_and_padding_zero":true,"rerun_bit_equal":true},"result":"PASS","summary":{"runs":1,"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.182832582155243e-05,"max_abs_dlogit":0.005308628082275391,"worst_run":"diag_l4k/text"},"by_source_arm":{"diag_long/text":{"runs":1,"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.182832582155243e-05,"max_abs_dlogit":0.005308628082275391,"worst_run":"diag_l4k/text"}},"head_bundle":{"dir":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","metadata_sha256":"631219aafcdb56ef1f008e5476f0e7a282933abdc51b4835132676d97124ab20","aimodelc":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_metadata":{"graph":"conversion/clef_flash/clef_head.py ClefHeadGraph: the author's JointSchemaHead for one record, tensor inputs only","shape":"bucket","functions":["t512","t1024","t2048","t4096"],"buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"dims":null,"host_pad":"bucket","host_pad_rule":"T to the smallest bucket >= T (512, 1024, 2048, 4096), Q to 16, O to 128; padding rows / columns zero, their valid flags 0","inputs":{"hidden":"[T, 4096] f32","key_valid":"[T] f32","q_avg":"[Q, T] f32","o_avg":"[O, T] f32","g_avg":"[1, T] f32","lexical":"[O, 4096] f32","member":"[Q, O] f32","type_ids":"[Q] i32","q_valid":"[Q] f32","o_valid":"[O] f32"},"input_order":["hidden","key_valid","q_avg","o_avg","g_avg","lexical","member","type_ids","q_valid","o_valid"],"outputs":{"logits":"[O] f32; padding options 0"},"weights":"fp16w32","softmax":"host: per question, fp32, over that question's options","lexical":"host: lm_head_fp16.bin rows of each option span's ids, averaged (clef_head.lexical_rows)","source":{"hf_id":"Cloudflare/clef-flash","revision":"17f0b0ad64efb65d273590632833508766b2aae6","joint_schema_model_sha256":"0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3","joint_head_safetensors_sha256":"19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0","joint_head_config_sha256":"77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0"}}},"descriptors":{"t512":{"function":"t512","inputs":{"hidden":[[512,4096],"float32"],"key_valid":[[512],"float32"],"q_avg":[[16,512],"float32"],"o_avg":[[128,512],"float32"],"g_avg":[[1,512],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t1024":{"function":"t1024","inputs":{"hidden":[[1024,4096],"float32"],"key_valid":[[1024],"float32"],"q_avg":[[16,1024],"float32"],"o_avg":[[128,1024],"float32"],"g_avg":[[1,1024],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t2048":{"function":"t2048","inputs":{"hidden":[[2048,4096],"float32"],"key_valid":[[2048],"float32"],"q_avg":[[16,2048],"float32"],"o_avg":[[128,2048],"float32"],"g_avg":[[1,2048],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t4096":{"function":"t4096","inputs":{"hidden":[[4096,4096],"float32"],"key_valid":[[4096],"float32"],"q_avg":[[16,4096],"float32"],"o_avg":[[128,4096],"float32"],"g_avg":[[1,4096],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}}},"descriptors_same_every_process":true,"oracle":{"path":"<lane>/oracle/diag_long/records_oracle.json","sha256":"a82663c78090b34e4b65ac91206fe81907ac3b4cc7fe7a43921f14ec4fc73ba0","runs":1},"table":{"path":"<lane>/exports/host/lm_head_fp16.bin","json":"<lane>/exports/host/lm_head_fp16.json"},"near_ties":[],"timing":{"contended":true,"by_T":{"T>1500":{"runs":1,"median_tokens":4050.0,"median_padded_T":4096.0,"first_call_ms_median":42.954708012985066,"second_call_ms_median":19.260166998719797,"host_ms_median":14.385291986400262}},"all_first_call_ms_median":42.954708012985066,"all_second_call_ms_median":19.260166998719797},"processes":[{"pid":26013,"started":1791004988.50686,"load_model_seconds":0.006020583998179063,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.32548333299928345,"t1024":0.001373584003886208,"t2048":0.0016910830163396895,"t4096":0.0010930410062428564},"reset_check":{"run":["diag_l4k","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791004988.97965,"process_wall_seconds":0.9375499580055475,"runs":1}],"red_arm":null,"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max"},"script":{"gate_head.py":"1d66d14945d23d15b80866799a6287a08fcca41cf95ff56ea41e418a8c0b5e0b","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d"},"seconds":{"gpu_processes":0.9389422079839278},"generated_at":"2026-10-03T05:23:09+00:00","runs":[{"id":"diag_l4k","arm":"text","variant":"base","tokens":4050,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":16,"layout":[[0,4],[4,8],[8,10],[10,13],[13,16]],"host_ms":14.385291986400262,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"diag_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006085634231567383,"mean_abs_dp":9.182832582155243e-05,"max_abs_dlogit":0.005308628082275391,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006085634231567383,"mean_abs_dp":9.183874499285594e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true}],"readout":{"path":"<lane>/results/readout_diag_long_fp16_pf64.json","sha256":"bc651ee470dbec4a2e1c19485766decdd0529194549723d747877d158a690b7e","bundle":"clef_flash_decode_fp16_pf64","result":"PASS","author_head_summary":{"max_abs_dp":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.183874499285594e-05,"argmax_equal_non_near_tie":5,"questions_non_near_tie":5,"argmax_equal_near_tie":0,"near_tie_questions":0},"vs_author_head_same_hidden":{"max_abs_dp":1.1920928955078125e-07,"argmax_equal_runs":1}}},"decide_py":{"fixture":{"schema":"clef-flash-decide-e2e/1","what":"decide.py from the raw request (host ids, host image preprocess + tower AOT, decoder AOT, head AOT, fp16 lm_head table) vs the author's fp32 systemone() response of the same record","assets":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","decoder_aot":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","head_bundle":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_aot":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"table":"<lane>/exports/host/lm_head_fp16.bin","table_sha256_recorded":"bfccf00b8d5d3f6d810839a52c2ec627700a9aaebaa26aedcb96931bd21ba36d","towers":{"256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"decoder_bundle_name":"clef_flash_decode_fp16_pf64","decoder_prefill_chunk":64},"load_seconds":{"decoder_s":2.3205162080121227,"head_s":0.06778220800333656,"tower_g256_s":0.1810511669900734,"tower_g448_s":0.18496129199047573},"oracle":"<lane>/oracle/records_oracle.json","fixtures":"<lane>/fixtures/records.json","summary":{"requests":12,"answer_values_equal":21,"answers":41,"fields_equal":30,"fields":163,"choices_equal":"18/18","max_abs_diff":0.008200000000000096,"ids_equal_all":true,"spans_equal_all":true,"model_and_usage_equal_all":true},"script_sha256":{"decide.py":"dd05d8481286810d19329e9f0d2b9cf93b391fec1884ffb038ef68622a207d71","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d","host.py":"356ae136ee62252a052a23bd13415228281065980754603aa89d75e225eb8e2a"},"timing_note":"contended GPU (other sessions); first request of the process includes warm-up","generated_at":"2026-10-03T04:58:36+00:00","runs":[{"id":"own_t01","arm":"text","source":"own_text","tokens":436,"wall_seconds":3.580666416004533,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9877003166559339,"response":{"model":"clef-flash","answers":{"department":{"type":"choice","choice":"technical","confidence":0.9783,"probabilities":{"billing":0.0097,"technical":0.9783,"shipping":0.0099,"account":0.0021}},"wants_refund":{"type":"noul","noul":0.0137},"urgency":{"type":"score","score":1.0889,"confidence":0.3807,"legend":{"0":"Can wait","1":"This week","2":"Today","3":"Immediately"},"probabilities":{"0":0.3085,"1":0.3807,"2":0.2241,"3":0.0867}}},"usage":{"input_tokens":436,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"department":{"type":"choice","choice":"technical","confidence":0.9782,"probabilities":{"billing":0.0097,"technical":0.9782,"shipping":0.01,"account":0.0021}},"wants_refund":{"type":"noul","noul":0.0136},"urgency":{"type":"score","score":1.0924,"confidence":0.381,"legend":{"0":"Can wait","1":"This week","2":"Today","3":"Immediately"},"probabilities":{"0":0.3069,"1":0.381,"2":0.2247,"3":0.0873}}},"usage":{"input_tokens":436,"output_tokens":0}},"compare":{"fields":12,"fields_equal":2,"answer_values":[{"question_id":"department","type":"choice","oracle":"technical","mine":"technical","equal":true},{"question_id":"wants_refund","type":"noul","oracle":0.0136,"mine":0.0137,"equal":false},{"question_id":"urgency","type":"score","oracle":1.0924,"mine":1.0889,"equal":false}],"answer_values_equal":1,"answers":3,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.0035000000000000586,"diffs":[{"question_id":"department","field":"confidence","oracle":0.9782,"mine":0.9783,"abs_diff":9.999999999998899e-05},{"question_id":"department","field":"p[technical]","oracle":0.9782,"mine":0.9783,"abs_diff":9.999999999998899e-05},{"question_id":"department","field":"p[shipping]","oracle":0.01,"mine":0.0099,"abs_diff":9.99999999999994e-05},{"question_id":"wants_refund","field":"noul","oracle":0.0136,"mine":0.0137,"abs_diff":0.00010000000000000113},{"question_id":"urgency","field":"score","oracle":1.0924,"mine":1.0889,"abs_diff":0.0035000000000000586},{"question_id":"urgency","field":"confidence","oracle":0.381,"mine":0.3807,"abs_diff":0.00030000000000002247},{"question_id":"urgency","field":"p[0]","oracle":0.3069,"mine":0.3085,"abs_diff":0.0015999999999999903},{"question_id":"urgency","field":"p[1]","oracle":0.381,"mine":0.3807,"abs_diff":0.00030000000000002247},{"question_id":"urgency","field":"p[2]","oracle":0.2247,"mine":0.2241,"abs_diff":0.0006000000000000172},{"question_id":"urgency","field":"p[3]","oracle":0.0873,"mine":0.0867,"abs_diff":0.0006000000000000033}]},"trace":{"host_ids_ms":1.0028749820776284,"tokens":436,"decoder":{"calls":7,"S":64,"call_ms":[2374.613624997437,132.9660839983262,133.70325000141747,133.93574999645352,133.8582499884069,134.00579101289622,134.27608300116844],"total_ms":3177.358832996106},"head":{"function":"t512","host_ms":1.514500007033348,"head_ms":21.145375008927658},"total_ms":3216.8169169744942}},{"id":"own_t03","arm":"text","source":"own_text","tokens":463,"wall_seconds":1.0875134590023663,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9913041881059135,"response":{"model":"clef-flash","answers":{"new_slot":{"type":"choice","choice":"tue_1400","confidence":0.9263,"probabilities":{"mon_0900":0.0121,"tue_1400":0.9263,"thu_1000":0.0616}},"thursday_still_on":{"type":"noul","noul":0.0872},"reply_needed":{"type":"noul","noul":0.7474}},"usage":{"input_tokens":463,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"new_slot":{"type":"choice","choice":"tue_1400","confidence":0.9257,"probabilities":{"mon_0900":0.0122,"tue_1400":0.9257,"thu_1000":0.0622}},"thursday_still_on":{"type":"noul","noul":0.0897},"reply_needed":{"type":"noul","noul":0.7556}},"usage":{"input_tokens":463,"output_tokens":0}},"compare":{"fields":6,"fields_equal":0,"answer_values":[{"question_id":"new_slot","type":"choice","oracle":"tue_1400","mine":"tue_1400","equal":true},{"question_id":"thursday_still_on","type":"noul","oracle":0.0897,"mine":0.0872,"equal":false},{"question_id":"reply_needed","type":"noul","oracle":0.7556,"mine":0.7474,"equal":false}],"answer_values_equal":1,"answers":3,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.008200000000000096,"diffs":[{"question_id":"new_slot","field":"confidence","oracle":0.9257,"mine":0.9263,"abs_diff":0.0006000000000000449},{"question_id":"new_slot","field":"p[mon_0900]","oracle":0.0122,"mine":0.0121,"abs_diff":0.00010000000000000113},{"question_id":"new_slot","field":"p[tue_1400]","oracle":0.9257,"mine":0.9263,"abs_diff":0.0006000000000000449},{"question_id":"new_slot","field":"p[thu_1000]","oracle":0.0622,"mine":0.0616,"abs_diff":0.0005999999999999964},{"question_id":"thursday_still_on","field":"noul","oracle":0.0897,"mine":0.0872,"abs_diff":0.0025000000000000022},{"question_id":"reply_needed","field":"noul","oracle":0.7556,"mine":0.7474,"abs_diff":0.008200000000000096}]},"trace":{"host_ids_ms":0.6413749943021685,"tokens":463,"decoder":{"calls":8,"S":64,"call_ms":[133.03979198099114,133.41233399114572,133.349834010005,133.4611659985967,133.526207995601,133.29416699707508,133.5389580053743,133.83358300779946],"total_ms":1067.4560419865884},"head":{"function":"t512","host_ms":0.8434999908786267,"head_ms":7.3741670057643205},"total_ms":1086.9082499993965}},{"id":"own_t15","arm":"text","source":"own_text_long","tokens":2603,"wall_seconds":5.5346896670234855,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.923740265229428,"response":{"model":"clef-flash","answers":{"non_renewal_notice":{"type":"choice","choice":"90","confidence":0.9885,"probabilities":{"30":0.0026,"60":0.0022,"90":0.9885,"120":0.0041,"none":0.0026}},"auto_renews":{"type":"noul","noul":0.9498},"liability_cap":{"type":"choice","choice":"fees_12m","confidence":0.9805,"probabilities":{"unlimited":0.0077,"fees_12m":0.9805,"fixed_amount":0.0066,"insurance":0.0052}},"assign_without_consent":{"type":"noul","noul":0.0175},"client_risk":{"type":"score","score":2.2984,"confidence":0.4662,"legend":{"0":"very unfavourable","1":"unfavourable","2":"balanced","3":"favourable","4":"very favourable"},"probabilities":{"0":0.0379,"1":0.0902,"2":0.4662,"3":0.3472,"4":0.0586}},"uptime_target":{"type":"choice","choice":"99.5","confidence":0.9809,"probabilities":{"99.0":0.0072,"99.5":0.9809,"99.9":0.0061,"99.99":0.0058}}},"usage":{"input_tokens":2603,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"non_renewal_notice":{"type":"choice","choice":"90","confidence":0.9886,"probabilities":{"30":0.0026,"60":0.0022,"90":0.9886,"120":0.0041,"none":0.0026}},"auto_renews":{"type":"noul","noul":0.9499},"liability_cap":{"type":"choice","choice":"fees_12m","confidence":0.9806,"probabilities":{"unlimited":0.0076,"fees_12m":0.9806,"fixed_amount":0.0065,"insurance":0.0052}},"assign_without_consent":{"type":"noul","noul":0.0174},"client_risk":{"type":"score","score":2.2981,"confidence":0.4667,"legend":{"0":"very unfavourable","1":"unfavourable","2":"balanced","3":"favourable","4":"very favourable"},"probabilities":{"0":0.0377,"1":0.0901,"2":0.4667,"3":0.3471,"4":0.0583}},"uptime_target":{"type":"choice","choice":"99.5","confidence":0.981,"probabilities":{"99.0":0.0071,"99.5":0.981,"99.9":0.0061,"99.99":0.0057}}},"usage":{"input_tokens":2603,"output_tokens":0}},"compare":{"fields":25,"fields_equal":6,"answer_values":[{"question_id":"non_renewal_notice","type":"choice","oracle":"90","mine":"90","equal":true},{"question_id":"auto_renews","type":"noul","oracle":0.9499,"mine":0.9498,"equal":false},{"question_id":"liability_cap","type":"choice","oracle":"fees_12m","mine":"fees_12m","equal":true},{"question_id":"assign_without_consent","type":"noul","oracle":0.0174,"mine":0.0175,"equal":false},{"question_id":"client_risk","type":"score","oracle":2.2981,"mine":2.2984,"equal":false},{"question_id":"uptime_target","type":"choice","oracle":"99.5","mine":"99.5","equal":true}],"answer_values_equal":3,"answers":6,"choices_equal":"3/3","model_equal":true,"usage_equal":true,"max_abs_diff":0.0005000000000000004,"diffs":[{"question_id":"non_renewal_notice","field":"confidence","oracle":0.9886,"mine":0.9885,"abs_diff":9.999999999998899e-05},{"question_id":"non_renewal_notice","field":"p[90]","oracle":0.9886,"mine":0.9885,"abs_diff":9.999999999998899e-05},{"question_id":"auto_renews","field":"noul","oracle":0.9499,"mine":0.9498,"abs_diff":9.999999999998899e-05},{"question_id":"liability_cap","field":"confidence","oracle":0.9806,"mine":0.9805,"abs_diff":9.999999999998899e-05},{"question_id":"liability_cap","field":"p[unlimited]","oracle":0.0076,"mine":0.0077,"abs_diff":0.00010000000000000026},{"question_id":"liability_cap","field":"p[fees_12m]","oracle":0.9806,"mine":0.9805,"abs_diff":9.999999999998899e-05},{"question_id":"liability_cap","field":"p[fixed_amount]","oracle":0.0065,"mine":0.0066,"abs_diff":0.00010000000000000026},{"question_id":"assign_without_consent","field":"noul","oracle":0.0174,"mine":0.0175,"abs_diff":0.00010000000000000286},{"question_id":"client_risk","field":"score","oracle":2.2981,"mine":2.2984,"abs_diff":0.000300000000000189},{"question_id":"client_risk","field":"confidence","oracle":0.4667,"mine":0.4662,"abs_diff":0.0005000000000000004},{"question_id":"client_risk","field":"p[0]","oracle":0.0377,"mine":0.0379,"abs_diff":0.00020000000000000573},{"question_id":"client_risk","field":"p[1]","oracle":0.0901,"mine":0.0902,"abs_diff":0.00010000000000000286},{"question_id":"client_risk","field":"p[2]","oracle":0.4667,"mine":0.4662,"abs_diff":0.0005000000000000004},{"question_id":"client_risk","field":"p[3]","oracle":0.3471,"mine":0.3472,"abs_diff":9.999999999998899e-05},{"question_id":"client_risk","field":"p[4]","oracle":0.0583,"mine":0.0586,"abs_diff":0.00030000000000000165},{"question_id":"uptime_target","field":"confidence","oracle":0.981,"mine":0.9809,"abs_diff":9.999999999998899e-05},{"question_id":"uptime_target","field":"p[99.0]","oracle":0.0071,"mine":0.0072,"abs_diff":9.99999999999994e-05},{"question_id":"uptime_target","field":"p[99.5]","oracle":0.981,"mine":0.9809,"abs_diff":9.999999999998899e-05},{"question_id":"uptime_target","field":"p[99.99]","oracle":0.0057,"mine":0.0058,"abs_diff":9.99999999999994e-05}]},"trace":{"host_ids_ms":2.7827910089399666,"tokens":2603,"decoder":{"calls":41,"S":64,"call_ms":[132.1790000074543,132.6080830185674,133.2199999887962,133.03262501722202,132.92412500595674,133.39229099801742,133.69700001203455,133.38770798873156,133.29183400492184,133.35808299598284,133.04058299399912,133.61045898636803,133.43204199918546,133.54358400101773,133.37670799228363,133.67979100439698,133.41258399304934,133.81441601086408,133.7749160011299,133.40920899645425,134.1442920092959,133.60345899127424,133.75012500910088,133.75587499467656,134.43037500837818,133.98679101374,134.16295900242403,133.82133399136364,134.25366699812002,134.14866698440164,134.509583003819,134.28920801379718,134.56066700746305,135.11716600623913,134.6537079953123,134.97370900586247,135.4060840094462,135.00095799099654,135.629083000822,135.20849999622442,134.81066699023359],"total_ms":5490.401918039424},"head":{"function":"t4096","host_ms":4.284833004930988,"head_ms":27.305124996928498},"total_ms":5534.094416012522}},{"id":"semif_a3f18f3a63d45345942b","arm":"text","source":"semif_authored144","tokens":185,"wall_seconds":0.41966916699311696,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9987879750988201,"response":{"model":"clef-flash","answers":{"decision":{"type":"choice","choice":"contradicted","confidence":0.9379,"probabilities":{"supported":0.032,"insufficient":0.0301,"contradicted":0.9379}}},"usage":{"input_tokens":185,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"decision":{"type":"choice","choice":"contradicted","confidence":0.938,"probabilities":{"supported":0.032,"insufficient":0.0301,"contradicted":0.938}}},"usage":{"input_tokens":185,"output_tokens":0}},"compare":{"fields":4,"fields_equal":2,"answer_values":[{"question_id":"decision","type":"choice","oracle":"contradicted","mine":"contradicted","equal":true}],"answer_values_equal":1,"answers":1,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":9.999999999998899e-05,"diffs":[{"question_id":"decision","field":"confidence","oracle":0.938,"mine":0.9379,"abs_diff":9.999999999998899e-05},{"question_id":"decision","field":"p[contradicted]","oracle":0.938,"mine":0.9379,"abs_diff":9.999999999998899e-05}]},"trace":{"host_ids_ms":0.36645898944698274,"tokens":185,"decoder":{"calls":3,"S":64,"call_ms":[132.68112501828,133.04474999313243,133.07604199508205],"total_ms":398.8019170064945},"head":{"function":"t512","host_ms":0.3914580156560987,"head_ms":9.898874995997176},"total_ms":418.998457986163}},{"id":"own_j01","arm":"text","source":"own_json","tokens":476,"wall_seconds":1.0856907500128727,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9994325119700979,"response":{"model":"clef-flash","answers":{"status":{"type":"choice","choice":"overdue","confidence":0.9024,"probabilities":{"paid":0.0122,"overdue":0.9024,"open_not_due":0.0733,"draft":0.0121}},"large":{"type":"noul","noul":0.0043},"priority":{"type":"score","score":1.1244,"confidence":0.4475,"legend":{"0":"no rush","1":"this month","2":"this week","3":"today"},"probabilities":{"0":0.2479,"1":0.4475,"2":0.2368,"3":0.0677}}},"usage":{"input_tokens":476,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"status":{"type":"choice","choice":"overdue","confidence":0.9028,"probabilities":{"paid":0.0123,"overdue":0.9028,"open_not_due":0.0728,"draft":0.0121}},"large":{"type":"noul","noul":0.0043},"priority":{"type":"score","score":1.1218,"confidence":0.4478,"legend":{"0":"no rush","1":"this month","2":"this week","3":"today"},"probabilities":{"0":0.249,"1":0.4478,"2":0.2356,"3":0.0676}}},"usage":{"input_tokens":476,"output_tokens":0}},"compare":{"fields":12,"fields_equal":2,"answer_values":[{"question_id":"status","type":"choice","oracle":"overdue","mine":"overdue","equal":true},{"question_id":"large","type":"noul","oracle":0.0043,"mine":0.0043,"equal":true},{"question_id":"priority","type":"score","oracle":1.1218,"mine":1.1244,"equal":false}],"answer_values_equal":2,"answers":3,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.0026000000000001577,"diffs":[{"question_id":"status","field":"confidence","oracle":0.9028,"mine":0.9024,"abs_diff":0.00040000000000006697},{"question_id":"status","field":"p[paid]","oracle":0.0123,"mine":0.0122,"abs_diff":9.99999999999994e-05},{"question_id":"status","field":"p[overdue]","oracle":0.9028,"mine":0.9024,"abs_diff":0.00040000000000006697},{"question_id":"status","field":"p[open_not_due]","oracle":0.0728,"mine":0.0733,"abs_diff":0.0005000000000000004},{"question_id":"priority","field":"score","oracle":1.1218,"mine":1.1244,"abs_diff":0.0026000000000001577},{"question_id":"priority","field":"confidence","oracle":0.4478,"mine":0.4475,"abs_diff":0.00029999999999996696},{"question_id":"priority","field":"p[0]","oracle":0.249,"mine":0.2479,"abs_diff":0.0010999999999999899},{"question_id":"priority","field":"p[1]","oracle":0.4478,"mine":0.4475,"abs_diff":0.00029999999999996696},{"question_id":"priority","field":"p[2]","oracle":0.2356,"mine":0.2368,"abs_diff":0.0012000000000000066},{"question_id":"priority","field":"p[3]","oracle":0.0676,"mine":0.0677,"abs_diff":0.00010000000000000286}]},"trace":{"host_ids_ms":0.700749980751425,"tokens":476,"decoder":{"calls":8,"S":64,"call_ms":[132.45470798574388,132.93191601405852,133.10816598823294,133.23041700641625,133.53779198951088,133.72579199494794,133.71866598026827,133.82395802182145],"total_ms":1066.5314149810001},"head":{"function":"t512","host_ms":0.6915839912835509,"head_ms":7.528165995609015},"total_ms":1084.9870840029325}},{"id":"own_j08","arm":"text","source":"own_json","tokens":473,"wall_seconds":1.0850730839883909,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9536699778247107,"response":{"model":"clef-flash","answers":{"risky":{"type":"noul","noul":0.7372},"riskiest_change":{"type":"choice","choice":"replicas","confidence":0.4871,"probabilities":{"replicas":0.4871,"log_level":0.1538,"db_pool_size":0.0074,"feature_flag":0.3517}},"review":{"type":"score","score":1.5772,"confidence":0.4564,"legend":{"0":"auto-approve","1":"peer review","2":"senior review","3":"block until a change window"},"probabilities":{"0":0.0834,"1":0.4564,"2":0.2599,"3":0.2003}}},"usage":{"input_tokens":473,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"risky":{"type":"noul","noul":0.7372},"riskiest_change":{"type":"choice","choice":"replicas","confidence":0.4872,"probabilities":{"replicas":0.4872,"log_level":0.1532,"db_pool_size":0.0074,"feature_flag":0.3522}},"review":{"type":"score","score":1.5774,"confidence":0.4566,"legend":{"0":"auto-approve","1":"peer review","2":"senior review","3":"block until a change window"},"probabilities":{"0":0.0835,"1":0.4566,"2":0.2589,"3":0.201}}},"usage":{"input_tokens":473,"output_tokens":0}},"compare":{"fields":12,"fields_equal":2,"answer_values":[{"question_id":"risky","type":"noul","oracle":0.7372,"mine":0.7372,"equal":true},{"question_id":"riskiest_change","type":"choice","oracle":"replicas","mine":"replicas","equal":true},{"question_id":"review","type":"score","oracle":1.5774,"mine":1.5772,"equal":false}],"answer_values_equal":2,"answers":3,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.0010000000000000009,"diffs":[{"question_id":"riskiest_change","field":"confidence","oracle":0.4872,"mine":0.4871,"abs_diff":0.0001000000000000445},{"question_id":"riskiest_change","field":"p[replicas]","oracle":0.4872,"mine":0.4871,"abs_diff":0.0001000000000000445},{"question_id":"riskiest_change","field":"p[log_level]","oracle":0.1532,"mine":0.1538,"abs_diff":0.0005999999999999894},{"question_id":"riskiest_change","field":"p[feature_flag]","oracle":0.3522,"mine":0.3517,"abs_diff":0.0005000000000000004},{"question_id":"review","field":"score","oracle":1.5774,"mine":1.5772,"abs_diff":0.00019999999999997797},{"question_id":"review","field":"confidence","oracle":0.4566,"mine":0.4564,"abs_diff":0.00020000000000003348},{"question_id":"review","field":"p[0]","oracle":0.0835,"mine":0.0834,"abs_diff":0.00010000000000000286},{"question_id":"review","field":"p[1]","oracle":0.4566,"mine":0.4564,"abs_diff":0.00020000000000003348},{"question_id":"review","field":"p[2]","oracle":0.2589,"mine":0.2599,"abs_diff":0.0010000000000000009},{"question_id":"review","field":"p[3]","oracle":0.201,"mine":0.2003,"abs_diff":0.0007000000000000062}]},"trace":{"host_ids_ms":0.6596249877475202,"tokens":473,"decoder":{"calls":8,"S":64,"call_ms":[132.43245799094439,133.20391700835899,132.9537080018781,133.50041600642726,133.4519590018317,133.8336250046268,133.41591699281707,133.639542007586],"total_ms":1066.4315420144703},"head":{"function":"t512","host_ms":0.755749992094934,"head_ms":7.4393340037204325},"total_ms":1084.4089580059517}},{"id":"own_j12","arm":"text","source":"own_json","tokens":562,"wall_seconds":1.2230669170094188,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9901558775620786,"response":{"model":"clef-flash","answers":{"coupon_valid":{"type":"noul","noul":0.0193},"total_band":{"type":"choice","choice":"10_to_20","confidence":0.7362,"probabilities":{"under_10":0.0859,"10_to_20":0.7362,"20_to_30":0.1472,"over_30":0.0307}},"dessert_items":{"type":"choice","choice":"1","confidence":0.5353,"probabilities":{"0":0.0481,"1":0.5353,"2":0.3709,"3":0.0458}},"basket_size":{"type":"score","score":0.8522,"confidence":0.8152,"legend":{"0":"small","1":"medium","2":"large"},"probabilities":{"0":0.1663,"1":0.8152,"2":0.0185}}},"usage":{"input_tokens":562,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"coupon_valid":{"type":"noul","noul":0.0192},"total_band":{"type":"choice","choice":"10_to_20","confidence":0.735,"probabilities":{"under_10":0.0861,"10_to_20":0.735,"20_to_30":0.148,"over_30":0.0309}},"dessert_items":{"type":"choice","choice":"1","confidence":0.5346,"probabilities":{"0":0.0482,"1":0.5346,"2":0.3715,"3":0.0457}},"basket_size":{"type":"score","score":0.8517,"confidence":0.8146,"legend":{"0":"small","1":"medium","2":"large"},"probabilities":{"0":0.1668,"1":0.8146,"2":0.0186}}},"usage":{"input_tokens":562,"output_tokens":0}},"compare":{"fields":16,"fields_equal":0,"answer_values":[{"question_id":"coupon_valid","type":"noul","oracle":0.0192,"mine":0.0193,"equal":false},{"question_id":"total_band","type":"choice","oracle":"10_to_20","mine":"10_to_20","equal":true},{"question_id":"dessert_items","type":"choice","oracle":"1","mine":"1","equal":true},{"question_id":"basket_size","type":"score","oracle":0.8517,"mine":0.8522,"equal":false}],"answer_values_equal":2,"answers":4,"choices_equal":"2/2","model_equal":true,"usage_equal":true,"max_abs_diff":0.0011999999999999789,"diffs":[{"question_id":"coupon_valid","field":"noul","oracle":0.0192,"mine":0.0193,"abs_diff":0.00010000000000000286},{"question_id":"total_band","field":"confidence","oracle":0.735,"mine":0.7362,"abs_diff":0.0011999999999999789},{"question_id":"total_band","field":"p[under_10]","oracle":0.0861,"mine":0.0859,"abs_diff":0.00019999999999999185},{"question_id":"total_band","field":"p[10_to_20]","oracle":0.735,"mine":0.7362,"abs_diff":0.0011999999999999789},{"question_id":"total_band","field":"p[20_to_30]","oracle":0.148,"mine":0.1472,"abs_diff":0.0007999999999999952},{"question_id":"total_band","field":"p[over_30]","oracle":0.0309,"mine":0.0307,"abs_diff":0.0001999999999999988},{"question_id":"dessert_items","field":"confidence","oracle":0.5346,"mine":0.5353,"abs_diff":0.0007000000000000339},{"question_id":"dessert_items","field":"p[0]","oracle":0.0482,"mine":0.0481,"abs_diff":0.00010000000000000286},{"question_id":"dessert_items","field":"p[1]","oracle":0.5346,"mine":0.5353,"abs_diff":0.0007000000000000339},{"question_id":"dessert_items","field":"p[2]","oracle":0.3715,"mine":0.3709,"abs_diff":0.0005999999999999894},{"question_id":"dessert_items","field":"p[3]","oracle":0.0457,"mine":0.0458,"abs_diff":0.00010000000000000286},{"question_id":"basket_size","field":"score","oracle":0.8517,"mine":0.8522,"abs_diff":0.0004999999999999449},{"question_id":"basket_size","field":"confidence","oracle":0.8146,"mine":0.8152,"abs_diff":0.0006000000000000449},{"question_id":"basket_size","field":"p[0]","oracle":0.1668,"mine":0.1663,"abs_diff":0.0005000000000000004},{"question_id":"basket_size","field":"p[1]","oracle":0.8146,"mine":0.8152,"abs_diff":0.0006000000000000449},{"question_id":"basket_size","field":"p[2]","oracle":0.0186,"mine":0.0185,"abs_diff":9.99999999999994e-05}]},"trace":{"host_ids_ms":1.1640830198302865,"tokens":562,"decoder":{"calls":9,"S":64,"call_ms":[132.39470799453557,132.75637500919402,132.98454202595167,133.39212501887232,134.16366701130755,133.86258398531936,133.6582080111839,133.771416003583,132.94345900067128],"total_ms":1199.9270840606187},"head":{"function":"t1024","host_ms":0.9200420172419399,"head_ms":10.746790998382494},"total_ms":1222.4175410228781}},{"id":"img_04","arm":"g448","source":"own_image","tokens":690,"wall_seconds":1.5917873750149738,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9857150305491842,"response":{"model":"clef-flash","answers":{"total":{"type":"choice","choice":"6.75","confidence":0.9582,"probabilities":{"5.75":0.017,"6.57":0.0131,"6.75":0.9582,"7.65":0.0117}},"paid_cash":{"type":"noul","noul":0.9679},"items":{"type":"choice","choice":"3","confidence":0.9654,"probabilities":{"1":0.006,"2":0.0083,"3":0.9654,"4":0.0088,"5":0.0059,"6":0.0057}},"legibility":{"type":"score","score":1.8184,"confidence":0.867,"legend":{"0":"illegible","1":"partly legible","2":"fully legible"},"probabilities":{"0":0.0486,"1":0.0845,"2":0.867}}},"usage":{"input_tokens":690,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"total":{"type":"choice","choice":"6.75","confidence":0.9583,"probabilities":{"5.75":0.0169,"6.57":0.0131,"6.75":0.9583,"7.65":0.0116}},"paid_cash":{"type":"noul","noul":0.9681},"items":{"type":"choice","choice":"3","confidence":0.9653,"probabilities":{"1":0.006,"2":0.0083,"3":0.9653,"4":0.0088,"5":0.0059,"6":0.0057}},"legibility":{"type":"score","score":1.8178,"confidence":0.8665,"legend":{"0":"illegible","1":"partly legible","2":"fully legible"},"probabilities":{"0":0.0487,"1":0.0848,"2":0.8665}}},"usage":{"input_tokens":690,"output_tokens":0}},"compare":{"fields":18,"fields_equal":6,"answer_values":[{"question_id":"total","type":"choice","oracle":"6.75","mine":"6.75","equal":true},{"question_id":"paid_cash","type":"noul","oracle":0.9681,"mine":0.9679,"equal":false},{"question_id":"items","type":"choice","oracle":"3","mine":"3","equal":true},{"question_id":"legibility","type":"score","oracle":1.8178,"mine":1.8184,"equal":false}],"answer_values_equal":2,"answers":4,"choices_equal":"2/2","model_equal":true,"usage_equal":true,"max_abs_diff":0.0005999999999999339,"diffs":[{"question_id":"total","field":"confidence","oracle":0.9583,"mine":0.9582,"abs_diff":9.999999999998899e-05},{"question_id":"total","field":"p[5.75]","oracle":0.0169,"mine":0.017,"abs_diff":0.00010000000000000286},{"question_id":"total","field":"p[6.75]","oracle":0.9583,"mine":0.9582,"abs_diff":9.999999999998899e-05},{"question_id":"total","field":"p[7.65]","oracle":0.0116,"mine":0.0117,"abs_diff":0.00010000000000000113},{"question_id":"paid_cash","field":"noul","oracle":0.9681,"mine":0.9679,"abs_diff":0.00019999999999997797},{"question_id":"items","field":"confidence","oracle":0.9653,"mine":0.9654,"abs_diff":9.999999999998899e-05},{"question_id":"items","field":"p[3]","oracle":0.9653,"mine":0.9654,"abs_diff":9.999999999998899e-05},{"question_id":"legibility","field":"score","oracle":1.8178,"mine":1.8184,"abs_diff":0.0005999999999999339},{"question_id":"legibility","field":"confidence","oracle":0.8665,"mine":0.867,"abs_diff":0.0004999999999999449},{"question_id":"legibility","field":"p[0]","oracle":0.0487,"mine":0.0486,"abs_diff":0.00010000000000000286},{"question_id":"legibility","field":"p[1]","oracle":0.0848,"mine":0.0845,"abs_diff":0.0002999999999999947},{"question_id":"legibility","field":"p[2]","oracle":0.8665,"mine":0.867,"abs_diff":0.0004999999999999449}]},"trace":{"tower":{"grid":448,"preprocess_ms":11.958083021454513,"tower_ms":90.75266699073836,"rows":196},"host_ids_ms":0.9633750014472753,"tokens":690,"decoder":{"calls":11,"S":64,"call_ms":[132.82954198075458,132.98516601207666,132.87775000208057,133.40200000675395,133.22512499871664,133.84683299227618,133.87950000469573,133.596083993325,132.8307919902727,133.24012499651872,133.30575000145473],"total_ms":1466.0186669789255},"head":{"function":"t1024","host_ms":0.9328339947387576,"head_ms":9.724041010485962},"total_ms":1591.1627910099924},"tower_min_row_cos_vs_oracle":0.9999999981708043},{"id":"img_06","arm":"g448","source":"own_image","tokens":597,"wall_seconds":1.436711499991361,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9829961555823702,"response":{"model":"clef-flash","answers":{"in_red_zone":{"type":"noul","noul":0.5098},"reading":{"type":"choice","choice":"75_100","confidence":0.5,"probabilities":{"0_25":0.0159,"25_50":0.0398,"50_75":0.4443,"75_100":0.5}},"alarm":{"type":"score","score":1.6358,"confidence":0.5686,"legend":{"0":"none","1":"advisory","2":"warning","3":"shutdown"},"probabilities":{"0":0.0995,"1":0.2486,"2":0.5686,"3":0.0833}}},"usage":{"input_tokens":597,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"in_red_zone":{"type":"noul","noul":0.5088},"reading":{"type":"choice","choice":"75_100","confidence":0.4981,"probabilities":{"0_25":0.0159,"25_50":0.0396,"50_75":0.4464,"75_100":0.4981}},"alarm":{"type":"score","score":1.6362,"confidence":0.5695,"legend":{"0":"none","1":"advisory","2":"warning","3":"shutdown"},"probabilities":{"0":0.099,"1":0.2486,"2":0.5695,"3":0.0828}}},"usage":{"input_tokens":597,"output_tokens":0}},"compare":{"fields":12,"fields_equal":2,"answer_values":[{"question_id":"in_red_zone","type":"noul","oracle":0.5088,"mine":0.5098,"equal":false},{"question_id":"reading","type":"choice","oracle":"75_100","mine":"75_100","equal":true},{"question_id":"alarm","type":"score","oracle":1.6362,"mine":1.6358,"equal":false}],"answer_values_equal":1,"answers":3,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.0021000000000000463,"diffs":[{"question_id":"in_red_zone","field":"noul","oracle":0.5088,"mine":0.5098,"abs_diff":0.0010000000000000009},{"question_id":"reading","field":"confidence","oracle":0.4981,"mine":0.5,"abs_diff":0.0019000000000000128},{"question_id":"reading","field":"p[25_50]","oracle":0.0396,"mine":0.0398,"abs_diff":0.0001999999999999988},{"question_id":"reading","field":"p[50_75]","oracle":0.4464,"mine":0.4443,"abs_diff":0.0021000000000000463},{"question_id":"reading","field":"p[75_100]","oracle":0.4981,"mine":0.5,"abs_diff":0.0019000000000000128},{"question_id":"alarm","field":"score","oracle":1.6362,"mine":1.6358,"abs_diff":0.000400000000000178},{"question_id":"alarm","field":"confidence","oracle":0.5695,"mine":0.5686,"abs_diff":0.0009000000000000119},{"question_id":"alarm","field":"p[0]","oracle":0.099,"mine":0.0995,"abs_diff":0.0005000000000000004},{"question_id":"alarm","field":"p[2]","oracle":0.5695,"mine":0.5686,"abs_diff":0.0009000000000000119},{"question_id":"alarm","field":"p[3]","oracle":0.0828,"mine":0.0833,"abs_diff":0.0005000000000000004}]},"trace":{"tower":{"grid":448,"preprocess_ms":2.2812499955762178,"tower_ms":80.00791701488197,"rows":196},"host_ids_ms":0.8237499860115349,"tokens":597,"decoder":{"calls":10,"S":64,"call_ms":[132.62237500748597,132.9328750143759,133.12595800380222,132.9821250110399,133.2666660018731,133.3348339830991,133.71145800920203,133.6616249755025,132.87008300540037,132.79766699997708],"total_ms":1331.3056660117581},"head":{"function":"t1024","host_ms":0.7835839933250099,"head_ms":9.802208020118997},"total_ms":1436.0435839917045},"tower_min_row_cos_vs_oracle":0.9999999976479518},{"id":"photo_02","arm":"g448","source":"cc0_photo","tokens":631,"wall_seconds":1.450415208004415,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.6312826424381595,"response":{"model":"clef-flash","answers":{"animal":{"type":"choice","choice":"cat","confidence":0.9617,"probabilities":{"cat":0.9617,"dog":0.0103,"rabbit":0.0102,"fox":0.0178}},"eye_color":{"type":"choice","choice":"yellow","confidence":0.9597,"probabilities":{"blue":0.007,"green":0.0212,"yellow":0.9597,"brown":0.0121}},"background":{"type":"score","score":0.3136,"confidence":0.7499,"legend":{"0":"plain","1":"somewhat busy","2":"busy"},"probabilities":{"0":0.7499,"1":0.1865,"2":0.0636}},"striped":{"type":"noul","noul":0.9319}},"usage":{"input_tokens":631,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"animal":{"type":"choice","choice":"cat","confidence":0.9616,"probabilities":{"cat":0.9616,"dog":0.0103,"rabbit":0.0102,"fox":0.0179}},"eye_color":{"type":"choice","choice":"yellow","confidence":0.9596,"probabilities":{"blue":0.0071,"green":0.0212,"yellow":0.9596,"brown":0.0121}},"background":{"type":"score","score":0.3142,"confidence":0.7494,"legend":{"0":"plain","1":"somewhat busy","2":"busy"},"probabilities":{"0":0.7494,"1":0.187,"2":0.0636}},"striped":{"type":"noul","noul":0.9319}},"usage":{"input_tokens":631,"output_tokens":0}},"compare":{"fields":16,"fields_equal":6,"answer_values":[{"question_id":"animal","type":"choice","oracle":"cat","mine":"cat","equal":true},{"question_id":"eye_color","type":"choice","oracle":"yellow","mine":"yellow","equal":true},{"question_id":"background","type":"score","oracle":0.3142,"mine":0.3136,"equal":false},{"question_id":"striped","type":"noul","oracle":0.9319,"mine":0.9319,"equal":true}],"answer_values_equal":3,"answers":4,"choices_equal":"2/2","model_equal":true,"usage_equal":true,"max_abs_diff":0.0005999999999999894,"diffs":[{"question_id":"animal","field":"confidence","oracle":0.9616,"mine":0.9617,"abs_diff":9.999999999998899e-05},{"question_id":"animal","field":"p[cat]","oracle":0.9616,"mine":0.9617,"abs_diff":9.999999999998899e-05},{"question_id":"animal","field":"p[fox]","oracle":0.0179,"mine":0.0178,"abs_diff":9.99999999999994e-05},{"question_id":"eye_color","field":"confidence","oracle":0.9596,"mine":0.9597,"abs_diff":9.999999999998899e-05},{"question_id":"eye_color","field":"p[blue]","oracle":0.0071,"mine":0.007,"abs_diff":0.00010000000000000026},{"question_id":"eye_color","field":"p[yellow]","oracle":0.9596,"mine":0.9597,"abs_diff":9.999999999998899e-05},{"question_id":"background","field":"score","oracle":0.3142,"mine":0.3136,"abs_diff":0.0005999999999999894},{"question_id":"background","field":"confidence","oracle":0.7494,"mine":0.7499,"abs_diff":0.000500000000000056},{"question_id":"background","field":"p[0]","oracle":0.7494,"mine":0.7499,"abs_diff":0.000500000000000056},{"question_id":"background","field":"p[1]","oracle":0.187,"mine":0.1865,"abs_diff":0.0005000000000000004}]},"trace":{"tower":{"grid":448,"preprocess_ms":11.29616700927727,"tower_ms":79.85841599293053,"rows":196},"host_ids_ms":0.9105410135816783,"tokens":631,"decoder":{"calls":10,"S":64,"call_ms":[132.99600000027567,133.36258300114423,133.3602079830598,133.23466602014378,133.4118750237394,133.96325000212528,133.8838750089053,133.76916700508446,133.56183297582902,133.44729100936092],"total_ms":1334.9907480296679},"head":{"function":"t1024","host_ms":0.8624579932074994,"head_ms":10.616707993904129},"total_ms":1449.7601250186563},"tower_min_row_cos_vs_oracle":0.9999999623569458},{"id":"img_01","arm":"g256","source":"own_image","tokens":430,"wall_seconds":0.9983807080134284,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9995646151203492,"response":{"model":"clef-flash","answers":{"circles":{"type":"choice","choice":"3","confidence":0.9391,"probabilities":{"1":0.0091,"2":0.0135,"3":0.9391,"4":0.0113,"5":0.0138,"6":0.0132}},"more_circles":{"type":"noul","noul":0.8073},"clutter":{"type":"score","score":0.5301,"confidence":0.491,"legend":{"0":"sparse","1":"moderate","2":"crowded"},"probabilities":{"0":0.491,"1":0.488,"2":0.0211}}},"usage":{"input_tokens":430,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"circles":{"type":"choice","choice":"3","confidence":0.9385,"probabilities":{"1":0.0092,"2":0.0136,"3":0.9385,"4":0.0114,"5":0.014,"6":0.0134}},"more_circles":{"type":"noul","noul":0.8058},"clutter":{"type":"score","score":0.5305,"confidence":0.4906,"legend":{"0":"sparse","1":"moderate","2":"crowded"},"probabilities":{"0":0.4906,"1":0.4884,"2":0.021}}},"usage":{"input_tokens":430,"output_tokens":0}},"compare":{"fields":13,"fields_equal":0,"answer_values":[{"question_id":"circles","type":"choice","oracle":"3","mine":"3","equal":true},{"question_id":"more_circles","type":"noul","oracle":0.8058,"mine":0.8073,"equal":false},{"question_id":"clutter","type":"score","oracle":0.5305,"mine":0.5301,"equal":false}],"answer_values_equal":1,"answers":3,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.0015000000000000568,"diffs":[{"question_id":"circles","field":"confidence","oracle":0.9385,"mine":0.9391,"abs_diff":0.0006000000000000449},{"question_id":"circles","field":"p[1]","oracle":0.0092,"mine":0.0091,"abs_diff":9.99999999999994e-05},{"question_id":"circles","field":"p[2]","oracle":0.0136,"mine":0.0135,"abs_diff":9.99999999999994e-05},{"question_id":"circles","field":"p[3]","oracle":0.9385,"mine":0.9391,"abs_diff":0.0006000000000000449},{"question_id":"circles","field":"p[4]","oracle":0.0114,"mine":0.0113,"abs_diff":0.00010000000000000113},{"question_id":"circles","field":"p[5]","oracle":0.014,"mine":0.0138,"abs_diff":0.00020000000000000052},{"question_id":"circles","field":"p[6]","oracle":0.0134,"mine":0.0132,"abs_diff":0.00020000000000000052},{"question_id":"more_circles","field":"noul","oracle":0.8058,"mine":0.8073,"abs_diff":0.0015000000000000568},{"question_id":"clutter","field":"score","oracle":0.5305,"mine":0.5301,"abs_diff":0.00039999999999995595},{"question_id":"clutter","field":"confidence","oracle":0.4906,"mine":0.491,"abs_diff":0.00040000000000001146},{"question_id":"clutter","field":"p[0]","oracle":0.4906,"mine":0.491,"abs_diff":0.00040000000000001146},{"question_id":"clutter","field":"p[1]","oracle":0.4884,"mine":0.488,"abs_diff":0.00040000000000001146},{"question_id":"clutter","field":"p[2]","oracle":0.021,"mine":0.0211,"abs_diff":9.99999999999994e-05}]},"trace":{"tower":{"grid":256,"preprocess_ms":1.151041011326015,"tower_ms":41.986666998127475,"rows":64},"host_ids_ms":0.5858750082552433,"tokens":430,"decoder":{"calls":7,"S":64,"call_ms":[132.90920900180936,132.94779197894968,132.88979200297035,132.97620799858123,133.79162500496022,134.3565419956576,134.03583300532773],"total_ms":933.9070009882562},"head":{"function":"t512","host_ms":0.6027919880580157,"head_ms":9.807499998714775},"total_ms":997.7062919933815},"tower_min_row_cos_vs_oracle":0.999999998720031},{"id":"img_08","arm":"g256","source":"own_image","tokens":522,"wall_seconds":1.2613047919876408,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9993569304845609,"response":{"model":"clef-flash","answers":{"largest_room":{"type":"choice","choice":"living","confidence":0.9279,"probabilities":{"living":0.9279,"kitchen":0.0246,"bed":0.0358,"bath":0.0117}},"rooms":{"type":"choice","choice":"5","confidence":0.7169,"probabilities":{"2":0.0125,"3":0.0366,"4":0.157,"5":0.7169,"6":0.0771}},"bath_next_to_bed":{"type":"noul","noul":0.3033},"complexity":{"type":"score","score":0.8442,"confidence":0.7234,"legend":{"0":"simple","1":"moderate","2":"complex"},"probabilities":{"0":0.2162,"1":0.7234,"2":0.0604}}},"usage":{"input_tokens":522,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"largest_room":{"type":"choice","choice":"living","confidence":0.9278,"probabilities":{"living":0.9278,"kitchen":0.0247,"bed":0.0358,"bath":0.0117}},"rooms":{"type":"choice","choice":"5","confidence":0.7153,"probabilities":{"2":0.0124,"3":0.0367,"4":0.1582,"5":0.7153,"6":0.0774}},"bath_next_to_bed":{"type":"noul","noul":0.3028},"complexity":{"type":"score","score":0.8446,"confidence":0.7235,"legend":{"0":"simple","1":"moderate","2":"complex"},"probabilities":{"0":0.216,"1":0.7235,"2":0.0606}}},"usage":{"input_tokens":522,"output_tokens":0}},"compare":{"fields":17,"fields_equal":2,"answer_values":[{"question_id":"largest_room","type":"choice","oracle":"living","mine":"living","equal":true},{"question_id":"rooms","type":"choice","oracle":"5","mine":"5","equal":true},{"question_id":"bath_next_to_bed","type":"noul","oracle":0.3028,"mine":0.3033,"equal":false},{"question_id":"complexity","type":"score","oracle":0.8446,"mine":0.8442,"equal":false}],"answer_values_equal":2,"answers":4,"choices_equal":"2/2","model_equal":true,"usage_equal":true,"max_abs_diff":0.0015999999999999348,"diffs":[{"question_id":"largest_room","field":"confidence","oracle":0.9278,"mine":0.9279,"abs_diff":9.999999999998899e-05},{"question_id":"largest_room","field":"p[living]","oracle":0.9278,"mine":0.9279,"abs_diff":9.999999999998899e-05},{"question_id":"largest_room","field":"p[kitchen]","oracle":0.0247,"mine":0.0246,"abs_diff":9.99999999999994e-05},{"question_id":"rooms","field":"confidence","oracle":0.7153,"mine":0.7169,"abs_diff":0.0015999999999999348},{"question_id":"rooms","field":"p[2]","oracle":0.0124,"mine":0.0125,"abs_diff":0.00010000000000000113},{"question_id":"rooms","field":"p[3]","oracle":0.0367,"mine":0.0366,"abs_diff":0.00010000000000000286},{"question_id":"rooms","field":"p[4]","oracle":0.1582,"mine":0.157,"abs_diff":0.0012000000000000066},{"question_id":"rooms","field":"p[5]","oracle":0.7153,"mine":0.7169,"abs_diff":0.0015999999999999348},{"question_id":"rooms","field":"p[6]","oracle":0.0774,"mine":0.0771,"abs_diff":0.0002999999999999947},{"question_id":"bath_next_to_bed","field":"noul","oracle":0.3028,"mine":0.3033,"abs_diff":0.0005000000000000004},{"question_id":"complexity","field":"score","oracle":0.8446,"mine":0.8442,"abs_diff":0.00040000000000006697},{"question_id":"complexity","field":"confidence","oracle":0.7235,"mine":0.7234,"abs_diff":9.999999999998899e-05},{"question_id":"complexity","field":"p[0]","oracle":0.216,"mine":0.2162,"abs_diff":0.00020000000000000573},{"question_id":"complexity","field":"p[1]","oracle":0.7235,"mine":0.7234,"abs_diff":9.999999999998899e-05},{"question_id":"complexity","field":"p[2]","oracle":0.0606,"mine":0.0604,"abs_diff":0.0001999999999999988}]},"trace":{"tower":{"grid":256,"preprocess_ms":4.035165999084711,"tower_ms":31.40174999134615,"rows":64},"host_ids_ms":0.7453330035787076,"tokens":522,"decoder":{"calls":9,"S":64,"call_ms":[132.94983300147578,132.83708397648297,133.3512089913711,133.02241699420847,133.94695898750797,133.97616700967774,133.90154202352278,133.9717500086408,133.51220899494365],"total_ms":1201.4691699878313},"head":{"function":"t1024","host_ms":0.8547089819330722,"head_ms":12.149665999459103},"total_ms":1260.6227500073146},"tower_min_row_cos_vs_oracle":0.9999999484457888}]},"heldout":{"schema":"clef-flash-decide-e2e/1","what":"decide.py from the raw request (host ids, host image preprocess + tower AOT, decoder AOT, head AOT, fp16 lm_head table) vs the author's fp32 systemone() response of the same record","assets":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","decoder_aot":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","head_bundle":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_aot":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"table":"<lane>/exports/host/lm_head_fp16.bin","table_sha256_recorded":"bfccf00b8d5d3f6d810839a52c2ec627700a9aaebaa26aedcb96931bd21ba36d","towers":{"256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"decoder_bundle_name":"clef_flash_decode_fp16_pf64","decoder_prefill_chunk":64},"load_seconds":{"decoder_s":20.79098791599972,"head_s":0.07257220899919048,"tower_g256_s":1.3939072089851834,"tower_g448_s":1.4347497080161702},"oracle":"<lane>/oracle/heldout/records_oracle.json","fixtures":"<lane>/fixtures/heldout/records.json","summary":{"requests":12,"answer_values_equal":36,"answers":60,"fields_equal":78,"fields":241,"choices_equal":"32/32","max_abs_diff":0.008199999999999985,"ids_equal_all":true,"spans_equal_all":true,"model_and_usage_equal_all":true},"script_sha256":{"decide.py":"dd05d8481286810d19329e9f0d2b9cf93b391fec1884ffb038ef68622a207d71","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d","host.py":"356ae136ee62252a052a23bd13415228281065980754603aa89d75e225eb8e2a"},"timing_note":"contended GPU (other sessions); first request of the process includes warm-up","generated_at":"2026-10-03T05:20:25+00:00","runs":[{"id":"ho_t01","arm":"text","source":"heldout_text","tokens":708,"wall_seconds":25.5221980829956,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9962073355757706,"response":{"model":"clef-flash","answers":{"leave_days":{"type":"choice","choice":"5","confidence":0.957,"probabilities":{"3":0.0119,"4":0.0144,"5":0.957,"6":0.0166}},"cover_arranged":{"type":"noul","noul":0.9347},"days_left_after":{"type":"choice","choice":"7","confidence":0.7863,"probabilities":{"5":0.0928,"7":0.7863,"12":0.0569,"17":0.064}},"reason":{"type":"choice","choice":"family_event","confidence":0.9869,"probabilities":{"family_event":0.9869,"medical":0.0046,"study":0.0045,"moving":0.004}},"ease_of_approval":{"type":"score","score":1.7316,"confidence":0.804,"legend":{"0":"hard","1":"needs a conversation","2":"straightforward"},"probabilities":{"0":0.0724,"1":0.1236,"2":0.804}}},"usage":{"input_tokens":708,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"leave_days":{"type":"choice","choice":"5","confidence":0.957,"probabilities":{"3":0.012,"4":0.0144,"5":0.957,"6":0.0167}},"cover_arranged":{"type":"noul","noul":0.9344},"days_left_after":{"type":"choice","choice":"7","confidence":0.7887,"probabilities":{"5":0.0915,"7":0.7887,"12":0.0563,"17":0.0635}},"reason":{"type":"choice","choice":"family_event","confidence":0.9869,"probabilities":{"family_event":0.9869,"medical":0.0046,"study":0.0045,"moving":0.004}},"ease_of_approval":{"type":"score","score":1.7315,"confidence":0.8039,"legend":{"0":"hard","1":"needs a conversation","2":"straightforward"},"probabilities":{"0":0.0724,"1":0.1237,"2":0.8039}}},"usage":{"input_tokens":708,"output_tokens":0}},"compare":{"fields":21,"fields_equal":9,"answer_values":[{"question_id":"leave_days","type":"choice","oracle":"5","mine":"5","equal":true},{"question_id":"cover_arranged","type":"noul","oracle":0.9344,"mine":0.9347,"equal":false},{"question_id":"days_left_after","type":"choice","oracle":"7","mine":"7","equal":true},{"question_id":"reason","type":"choice","oracle":"family_event","mine":"family_event","equal":true},{"question_id":"ease_of_approval","type":"score","oracle":1.7315,"mine":1.7316,"equal":false}],"answer_values_equal":3,"answers":5,"choices_equal":"3/3","model_equal":true,"usage_equal":true,"max_abs_diff":0.0023999999999999577,"diffs":[{"question_id":"leave_days","field":"p[3]","oracle":0.012,"mine":0.0119,"abs_diff":9.99999999999994e-05},{"question_id":"leave_days","field":"p[6]","oracle":0.0167,"mine":0.0166,"abs_diff":9.99999999999994e-05},{"question_id":"cover_arranged","field":"noul","oracle":0.9344,"mine":0.9347,"abs_diff":0.00029999999999996696},{"question_id":"days_left_after","field":"confidence","oracle":0.7887,"mine":0.7863,"abs_diff":0.0023999999999999577},{"question_id":"days_left_after","field":"p[5]","oracle":0.0915,"mine":0.0928,"abs_diff":0.0012999999999999956},{"question_id":"days_left_after","field":"p[7]","oracle":0.7887,"mine":0.7863,"abs_diff":0.0023999999999999577},{"question_id":"days_left_after","field":"p[12]","oracle":0.0563,"mine":0.0569,"abs_diff":0.0005999999999999964},{"question_id":"days_left_after","field":"p[17]","oracle":0.0635,"mine":0.064,"abs_diff":0.0005000000000000004},{"question_id":"ease_of_approval","field":"score","oracle":1.7315,"mine":1.7316,"abs_diff":9.999999999998899e-05},{"question_id":"ease_of_approval","field":"confidence","oracle":0.8039,"mine":0.804,"abs_diff":0.00010000000000010001},{"question_id":"ease_of_approval","field":"p[1]","oracle":0.1237,"mine":0.1236,"abs_diff":0.00010000000000000286},{"question_id":"ease_of_approval","field":"p[2]","oracle":0.8039,"mine":0.804,"abs_diff":0.00010000000000010001}]},"trace":{"host_ids_ms":1.2612919963430613,"tokens":708,"decoder":{"calls":12,"S":64,"call_ms":[23650.77208299772,133.24345799628645,133.93587499740534,133.89600001391955,134.47020898456685,134.55991601222195,134.53199999639764,134.43029200425372,134.36645801994018,134.3795409775339,134.3943749961909,133.98608399438672],"total_ms":25126.966290990822},"head":{"function":"t1024","host_ms":1.3350409863051027,"head_ms":21.894083998631686},"total_ms":25167.10741701536}},{"id":"ho_t05","arm":"text","source":"heldout_text","tokens":727,"wall_seconds":1.633697207988007,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9965155174791962,"response":{"model":"clef-flash","answers":{"defect_kind":{"type":"choice","choice":"crash","confidence":0.9176,"probabilities":{"crash":0.9176,"display":0.0289,"sync":0.0127,"slowness":0.0408}},"regression":{"type":"noul","noul":0.9449},"loses_work":{"type":"noul","noul":0.9562},"small_notebooks":{"type":"noul","noul":0.0081},"priority":{"type":"score","score":1.8845,"confidence":0.6309,"legend":{"0":"P4 backlog","1":"P3 normal","2":"P2 high","3":"P1 urgent"},"probabilities":{"0":0.0641,"1":0.1461,"2":0.6309,"3":0.1588}}},"usage":{"input_tokens":727,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"defect_kind":{"type":"choice","choice":"crash","confidence":0.9177,"probabilities":{"crash":0.9177,"display":0.0288,"sync":0.0127,"slowness":0.0408}},"regression":{"type":"noul","noul":0.9448},"loses_work":{"type":"noul","noul":0.9563},"small_notebooks":{"type":"noul","noul":0.0081},"priority":{"type":"score","score":1.8841,"confidence":0.6307,"legend":{"0":"P4 backlog","1":"P3 normal","2":"P2 high","3":"P1 urgent"},"probabilities":{"0":0.0644,"1":0.146,"2":0.6307,"3":0.1589}}},"usage":{"input_tokens":727,"output_tokens":0}},"compare":{"fields":14,"fields_equal":3,"answer_values":[{"question_id":"defect_kind","type":"choice","oracle":"crash","mine":"crash","equal":true},{"question_id":"regression","type":"noul","oracle":0.9448,"mine":0.9449,"equal":false},{"question_id":"loses_work","type":"noul","oracle":0.9563,"mine":0.9562,"equal":false},{"question_id":"small_notebooks","type":"noul","oracle":0.0081,"mine":0.0081,"equal":true},{"question_id":"priority","type":"score","oracle":1.8841,"mine":1.8845,"equal":false}],"answer_values_equal":2,"answers":5,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.00039999999999995595,"diffs":[{"question_id":"defect_kind","field":"confidence","oracle":0.9177,"mine":0.9176,"abs_diff":9.999999999998899e-05},{"question_id":"defect_kind","field":"p[crash]","oracle":0.9177,"mine":0.9176,"abs_diff":9.999999999998899e-05},{"question_id":"defect_kind","field":"p[display]","oracle":0.0288,"mine":0.0289,"abs_diff":9.99999999999994e-05},{"question_id":"regression","field":"noul","oracle":0.9448,"mine":0.9449,"abs_diff":9.999999999998899e-05},{"question_id":"loses_work","field":"noul","oracle":0.9563,"mine":0.9562,"abs_diff":9.999999999998899e-05},{"question_id":"priority","field":"score","oracle":1.8841,"mine":1.8845,"abs_diff":0.00039999999999995595},{"question_id":"priority","field":"confidence","oracle":0.6307,"mine":0.6309,"abs_diff":0.00019999999999997797},{"question_id":"priority","field":"p[0]","oracle":0.0644,"mine":0.0641,"abs_diff":0.0002999999999999947},{"question_id":"priority","field":"p[1]","oracle":0.146,"mine":0.1461,"abs_diff":0.00010000000000001674},{"question_id":"priority","field":"p[2]","oracle":0.6307,"mine":0.6309,"abs_diff":0.00019999999999997797},{"question_id":"priority","field":"p[3]","oracle":0.1589,"mine":0.1588,"abs_diff":0.00010000000000001674}]},"trace":{"host_ids_ms":1.0072079894598573,"tokens":727,"decoder":{"calls":12,"S":64,"call_ms":[133.49279199610464,133.99462500819936,133.81037500221282,133.83250002516434,134.06570799998008,134.84379200963303,134.7061249834951,134.53691601171158,134.38062500790693,134.22187502146699,134.16616700123996,134.37074999092147],"total_ms":1610.4222500580363},"head":{"function":"t1024","host_ms":1.1475840001367033,"head_ms":9.97833299334161},"total_ms":1632.8758750169072}},{"id":"ho_t07","arm":"text","source":"heldout_text","tokens":1086,"wall_seconds":2.312755416991422,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9948426124176317,"response":{"model":"clef-flash","answers":{"factor":{"type":"choice","choice":"3","confidence":0.8423,"probabilities":{"1.5":0.055,"2":0.0782,"3":0.8423,"4":0.0141,"6":0.0103}},"oats_total":{"type":"choice","choice":"750","confidence":0.9123,"probabilities":{"500":0.0059,"750":0.9123,"1000":0.0638,"1250":0.018}},"syrup_total":{"type":"choice","choice":"240","confidence":0.9515,"probabilities":{"60":0.0053,"80":0.0092,"240":0.9515,"320":0.034}},"oats_one_weighing":{"type":"noul","noul":0.253},"asks_swap":{"type":"noul","noul":0.9758},"oven_temperature":{"type":"choice","choice":"180","confidence":0.9766,"probabilities":{"160":0.0078,"170":0.0076,"180":0.9766,"200":0.008}},"questions_asked":{"type":"choice","choice":"2","confidence":0.404,"probabilities":{"1":0.0098,"2":0.404,"3":0.4011,"4":0.1851}},"difficulty":{"type":"score","score":1.9497,"confidence":0.7689,"legend":{"0":"trivial","1":"easy","2":"moderate","3":"tricky"},"probabilities":{"0":0.0382,"1":0.0834,"2":0.7689,"3":0.1095}}},"usage":{"input_tokens":1086,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"factor":{"type":"choice","choice":"3","confidence":0.8409,"probabilities":{"1.5":0.0554,"2":0.0792,"3":0.8409,"4":0.0142,"6":0.0104}},"oats_total":{"type":"choice","choice":"750","confidence":0.9122,"probabilities":{"500":0.006,"750":0.9122,"1000":0.0638,"1250":0.018}},"syrup_total":{"type":"choice","choice":"240","confidence":0.9516,"probabilities":{"60":0.0054,"80":0.0093,"240":0.9516,"320":0.0338}},"oats_one_weighing":{"type":"noul","noul":0.2547},"asks_swap":{"type":"noul","noul":0.9759},"oven_temperature":{"type":"choice","choice":"180","confidence":0.9765,"probabilities":{"160":0.0078,"170":0.0076,"180":0.9765,"200":0.0081}},"questions_asked":{"type":"choice","choice":"2","confidence":0.4102,"probabilities":{"1":0.0099,"2":0.4102,"3":0.3985,"4":0.1815}},"difficulty":{"type":"score","score":1.9475,"confidence":0.7682,"legend":{"0":"trivial","1":"easy","2":"moderate","3":"tricky"},"probabilities":{"0":0.0386,"1":0.0842,"2":0.7682,"3":0.109}}},"usage":{"input_tokens":1086,"output_tokens":0}},"compare":{"fields":34,"fields_equal":4,"answer_values":[{"question_id":"factor","type":"choice","oracle":"3","mine":"3","equal":true},{"question_id":"oats_total","type":"choice","oracle":"750","mine":"750","equal":true},{"question_id":"syrup_total","type":"choice","oracle":"240","mine":"240","equal":true},{"question_id":"oats_one_weighing","type":"noul","oracle":0.2547,"mine":0.253,"equal":false},{"question_id":"asks_swap","type":"noul","oracle":0.9759,"mine":0.9758,"equal":false},{"question_id":"oven_temperature","type":"choice","oracle":"180","mine":"180","equal":true},{"question_id":"questions_asked","type":"choice","oracle":"2","mine":"2","equal":true},{"question_id":"difficulty","type":"score","oracle":1.9475,"mine":1.9497,"equal":false}],"answer_values_equal":5,"answers":8,"choices_equal":"5/5","model_equal":true,"usage_equal":true,"max_abs_diff":0.006199999999999983,"diffs":[{"question_id":"factor","field":"confidence","oracle":0.8409,"mine":0.8423,"abs_diff":0.0014000000000000679},{"question_id":"factor","field":"p[1.5]","oracle":0.0554,"mine":0.055,"abs_diff":0.0003999999999999976},{"question_id":"factor","field":"p[2]","oracle":0.0792,"mine":0.0782,"abs_diff":0.0010000000000000009},{"question_id":"factor","field":"p[3]","oracle":0.8409,"mine":0.8423,"abs_diff":0.0014000000000000679},{"question_id":"factor","field":"p[4]","oracle":0.0142,"mine":0.0141,"abs_diff":0.00010000000000000113},{"question_id":"factor","field":"p[6]","oracle":0.0104,"mine":0.0103,"abs_diff":9.99999999999994e-05},{"question_id":"oats_total","field":"confidence","oracle":0.9122,"mine":0.9123,"abs_diff":9.999999999998899e-05},{"question_id":"oats_total","field":"p[500]","oracle":0.006,"mine":0.0059,"abs_diff":0.00010000000000000026},{"question_id":"oats_total","field":"p[750]","oracle":0.9122,"mine":0.9123,"abs_diff":9.999999999998899e-05},{"question_id":"syrup_total","field":"confidence","oracle":0.9516,"mine":0.9515,"abs_diff":9.999999999998899e-05},{"question_id":"syrup_total","field":"p[60]","oracle":0.0054,"mine":0.0053,"abs_diff":0.00010000000000000026},{"question_id":"syrup_total","field":"p[80]","oracle":0.0093,"mine":0.0092,"abs_diff":9.99999999999994e-05},{"question_id":"syrup_total","field":"p[240]","oracle":0.9516,"mine":0.9515,"abs_diff":9.999999999998899e-05},{"question_id":"syrup_total","field":"p[320]","oracle":0.0338,"mine":0.034,"abs_diff":0.00020000000000000573},{"question_id":"oats_one_weighing","field":"noul","oracle":0.2547,"mine":0.253,"abs_diff":0.0016999999999999793},{"question_id":"asks_swap","field":"noul","oracle":0.9759,"mine":0.9758,"abs_diff":9.999999999998899e-05},{"question_id":"oven_temperature","field":"confidence","oracle":0.9765,"mine":0.9766,"abs_diff":9.999999999998899e-05},{"question_id":"oven_temperature","field":"p[180]","oracle":0.9765,"mine":0.9766,"abs_diff":9.999999999998899e-05},{"question_id":"oven_temperature","field":"p[200]","oracle":0.0081,"mine":0.008,"abs_diff":9.99999999999994e-05},{"question_id":"questions_asked","field":"confidence","oracle":0.4102,"mine":0.404,"abs_diff":0.006199999999999983},{"question_id":"questions_asked","field":"p[1]","oracle":0.0099,"mine":0.0098,"abs_diff":0.00010000000000000113},{"question_id":"questions_asked","field":"p[2]","oracle":0.4102,"mine":0.404,"abs_diff":0.006199999999999983},{"question_id":"questions_asked","field":"p[3]","oracle":0.3985,"mine":0.4011,"abs_diff":0.002599999999999991},{"question_id":"questions_asked","field":"p[4]","oracle":0.1815,"mine":0.1851,"abs_diff":0.003599999999999992},{"question_id":"difficulty","field":"score","oracle":1.9475,"mine":1.9497,"abs_diff":0.0021999999999999797},{"question_id":"difficulty","field":"confidence","oracle":0.7682,"mine":0.7689,"abs_diff":0.0007000000000000339},{"question_id":"difficulty","field":"p[0]","oracle":0.0386,"mine":0.0382,"abs_diff":0.0004000000000000045},{"question_id":"difficulty","field":"p[1]","oracle":0.0842,"mine":0.0834,"abs_diff":0.0007999999999999952},{"question_id":"difficulty","field":"p[2]","oracle":0.7682,"mine":0.7689,"abs_diff":0.0007000000000000339},{"question_id":"difficulty","field":"p[3]","oracle":0.109,"mine":0.1095,"abs_diff":0.0005000000000000004}]},"trace":{"host_ids_ms":1.2337079970166087,"tokens":1086,"decoder":{"calls":17,"S":64,"call_ms":[133.74916699831374,133.69024998974055,134.0952919854317,134.1777500056196,134.22354098293,134.16704200790264,134.7011250036303,134.69695899402723,133.8323340169154,134.06341700465418,134.56229199073277,134.86779198865406,134.25820801057853,134.178541979054,134.40270800492726,134.07933301641606,134.34150000102818],"total_ms":2282.087251980556},"head":{"function":"t2048","host_ms":3.062542004045099,"head_ms":15.779332985403016},"total_ms":2311.9862499879673}},{"id":"ho_l01","arm":"text","source":"heldout_text_long","tokens":2850,"wall_seconds":6.120023291994585,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9802225236587327,"response":{"model":"clef-flash","answers":{"root_cause":{"type":"choice","choice":"contactor","confidence":0.976,"probabilities":{"contactor":0.976,"door":0.0111,"power_cut":0.0042,"refrigerant_leak":0.0043,"sensor":0.0044}},"peak_temperature":{"type":"choice","choice":"-12.4","confidence":0.9863,"probabilities":{"-18.0":0.0033,"-14.9":0.0076,"-12.4":0.9863,"-9.6":0.0027}},"first_text_reached":{"type":"noul","noul":0.0371},"dough_outcome":{"type":"choice","choice":"held","confidence":0.9856,"probabilities":{"released":0.0035,"destroyed":0.0041,"held":0.9856,"left_in_chamber_3":0.0068}},"standby_ready":{"type":"noul","noul":0.0156},"above_limit":{"type":"choice","choice":"3_to_6h","confidence":0.4689,"probabilities":{"under_1h":0.0404,"1_to_3h":0.1734,"3_to_6h":0.4689,"over_6h":0.3173}},"seriousness":{"type":"score","score":1.513,"confidence":0.4707,"legend":{"0":"minor","1":"moderate","2":"serious","3":"critical"},"probabilities":{"0":0.0378,"1":0.4707,"2":0.4321,"3":0.0594}}},"usage":{"input_tokens":2850,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"root_cause":{"type":"choice","choice":"contactor","confidence":0.9761,"probabilities":{"contactor":0.9761,"door":0.011,"power_cut":0.0042,"refrigerant_leak":0.0043,"sensor":0.0044}},"peak_temperature":{"type":"choice","choice":"-12.4","confidence":0.9864,"probabilities":{"-18.0":0.0033,"-14.9":0.0076,"-12.4":0.9864,"-9.6":0.0027}},"first_text_reached":{"type":"noul","noul":0.0369},"dough_outcome":{"type":"choice","choice":"held","confidence":0.9856,"probabilities":{"released":0.0035,"destroyed":0.0041,"held":0.9856,"left_in_chamber_3":0.0068}},"standby_ready":{"type":"noul","noul":0.0156},"above_limit":{"type":"choice","choice":"3_to_6h","confidence":0.4689,"probabilities":{"under_1h":0.0407,"1_to_3h":0.1743,"3_to_6h":0.4689,"over_6h":0.3161}},"seriousness":{"type":"score","score":1.5141,"confidence":0.4698,"legend":{"0":"minor","1":"moderate","2":"serious","3":"critical"},"probabilities":{"0":0.0378,"1":0.4698,"2":0.433,"3":0.0594}}},"usage":{"input_tokens":2850,"output_tokens":0}},"compare":{"fields":29,"fields_equal":16,"answer_values":[{"question_id":"root_cause","type":"choice","oracle":"contactor","mine":"contactor","equal":true},{"question_id":"peak_temperature","type":"choice","oracle":"-12.4","mine":"-12.4","equal":true},{"question_id":"first_text_reached","type":"noul","oracle":0.0369,"mine":0.0371,"equal":false},{"question_id":"dough_outcome","type":"choice","oracle":"held","mine":"held","equal":true},{"question_id":"standby_ready","type":"noul","oracle":0.0156,"mine":0.0156,"equal":true},{"question_id":"above_limit","type":"choice","oracle":"3_to_6h","mine":"3_to_6h","equal":true},{"question_id":"seriousness","type":"score","oracle":1.5141,"mine":1.513,"equal":false}],"answer_values_equal":5,"answers":7,"choices_equal":"4/4","model_equal":true,"usage_equal":true,"max_abs_diff":0.0012000000000000344,"diffs":[{"question_id":"root_cause","field":"confidence","oracle":0.9761,"mine":0.976,"abs_diff":9.999999999998899e-05},{"question_id":"root_cause","field":"p[contactor]","oracle":0.9761,"mine":0.976,"abs_diff":9.999999999998899e-05},{"question_id":"root_cause","field":"p[door]","oracle":0.011,"mine":0.0111,"abs_diff":0.00010000000000000113},{"question_id":"peak_temperature","field":"confidence","oracle":0.9864,"mine":0.9863,"abs_diff":0.00010000000000010001},{"question_id":"peak_temperature","field":"p[-12.4]","oracle":0.9864,"mine":0.9863,"abs_diff":0.00010000000000010001},{"question_id":"first_text_reached","field":"noul","oracle":0.0369,"mine":0.0371,"abs_diff":0.0001999999999999988},{"question_id":"above_limit","field":"p[under_1h]","oracle":0.0407,"mine":0.0404,"abs_diff":0.00030000000000000165},{"question_id":"above_limit","field":"p[1_to_3h]","oracle":0.1743,"mine":0.1734,"abs_diff":0.0009000000000000119},{"question_id":"above_limit","field":"p[over_6h]","oracle":0.3161,"mine":0.3173,"abs_diff":0.0012000000000000344},{"question_id":"seriousness","field":"score","oracle":1.5141,"mine":1.513,"abs_diff":0.001100000000000101},{"question_id":"seriousness","field":"confidence","oracle":0.4698,"mine":0.4707,"abs_diff":0.0009000000000000119},{"question_id":"seriousness","field":"p[1]","oracle":0.4698,"mine":0.4707,"abs_diff":0.0009000000000000119},{"question_id":"seriousness","field":"p[2]","oracle":0.433,"mine":0.4321,"abs_diff":0.0009000000000000119}]},"trace":{"host_ids_ms":2.8744160081259906,"tokens":2850,"decoder":{"calls":45,"S":64,"call_ms":[133.43395799165592,133.96262499736622,134.02850000420585,134.34045799658634,134.67479197424836,133.90170800266787,134.33245799387805,134.75266701425426,134.08679200801998,133.68687499314547,133.704999985639,133.9452080137562,134.3432089779526,134.42616700194776,134.6026669780258,135.18874999135733,134.86470800125971,134.53937499434687,134.64433298213407,134.66491698636673,134.75774999824353,134.99912500265054,134.8782499844674,134.98416700167581,135.0923749851063,134.791916992981,135.5818749871105,134.91104199783877,135.61495899921283,135.1407080073841,135.389834002126,135.59404198895209,135.28737501474097,135.47833400662057,135.7998749881517,135.31716700526886,136.09270798042417,135.63720899401233,136.26312499400228,135.97170801949687,135.766416991828,136.2119999830611,136.49641699157655,135.87020899285562,136.291708011413],"total_ms":6074.345462810015},"head":{"function":"t4096","host_ms":5.363417003536597,"head_ms":27.075541991507635},"total_ms":6119.329583016224}},{"id":"ho_j02","arm":"text","source":"heldout_json","tokens":335,"wall_seconds":0.8269030419760384,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9988922547222996,"response":{"model":"clef-flash","answers":{"beyond_tolerance":{"type":"choice","choice":"3","confidence":0.466,"probabilities":{"0":0.0241,"1":0.0643,"2":0.3678,"3":0.466,"4":0.0586,"5":0.0192}}},"usage":{"input_tokens":335,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"beyond_tolerance":{"type":"choice","choice":"3","confidence":0.4671,"probabilities":{"0":0.024,"1":0.064,"2":0.3669,"3":0.4671,"4":0.0588,"5":0.0192}}},"usage":{"input_tokens":335,"output_tokens":0}},"compare":{"fields":7,"fields_equal":1,"answer_values":[{"question_id":"beyond_tolerance","type":"choice","oracle":"3","mine":"3","equal":true}],"answer_values_equal":1,"answers":1,"choices_equal":"1/1","model_equal":true,"usage_equal":true,"max_abs_diff":0.0010999999999999899,"diffs":[{"question_id":"beyond_tolerance","field":"confidence","oracle":0.4671,"mine":0.466,"abs_diff":0.0010999999999999899},{"question_id":"beyond_tolerance","field":"p[0]","oracle":0.024,"mine":0.0241,"abs_diff":9.99999999999994e-05},{"question_id":"beyond_tolerance","field":"p[1]","oracle":0.064,"mine":0.0643,"abs_diff":0.0002999999999999947},{"question_id":"beyond_tolerance","field":"p[2]","oracle":0.3669,"mine":0.3678,"abs_diff":0.0009000000000000119},{"question_id":"beyond_tolerance","field":"p[3]","oracle":0.4671,"mine":0.466,"abs_diff":0.0010999999999999899},{"question_id":"beyond_tolerance","field":"p[4]","oracle":0.0588,"mine":0.0586,"abs_diff":0.0001999999999999988}]},"trace":{"host_ids_ms":0.47675002133473754,"tokens":335,"decoder":{"calls":6,"S":64,"call_ms":[134.33287499356084,133.89454199932516,134.2056669818703,134.4245000218507,134.00900000124238,134.63808299275115],"total_ms":805.5046669906005},"head":{"function":"t512","host_ms":0.8848329889588058,"head_ms":9.615917020710185},"total_ms":826.1569169990253}},{"id":"ho_j06","arm":"text","source":"heldout_json","tokens":733,"wall_seconds":1.6373670829925686,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9796648257196925,"response":{"model":"clef-flash","answers":{"hides_endpoints":{"type":"noul","noul":0.0167},"route_audience":{"type":"choice","choice":"followers","confidence":0.9838,"probabilities":{"everyone":0.0108,"followers":0.9838,"nobody":0.0054}},"marketing":{"type":"noul","noul":0.0045},"zone_boundaries":{"type":"choice","choice":"3","confidence":0.6354,"probabilities":{"3":0.6354,"4":0.2693,"5":0.0551,"6":0.0402}},"unit_mismatch":{"type":"noul","noul":0.3013},"privacy_level":{"type":"score","score":1.2756,"confidence":0.4661,"legend":{"0":"very open","1":"open","2":"moderate","3":"cautious","4":"very cautious"},"probabilities":{"0":0.1468,"1":0.4661,"2":0.357,"3":0.0247,"4":0.0054}}},"usage":{"input_tokens":733,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"hides_endpoints":{"type":"noul","noul":0.0168},"route_audience":{"type":"choice","choice":"followers","confidence":0.9838,"probabilities":{"everyone":0.0108,"followers":0.9838,"nobody":0.0054}},"marketing":{"type":"noul","noul":0.0045},"zone_boundaries":{"type":"choice","choice":"3","confidence":0.6436,"probabilities":{"3":0.6436,"4":0.2612,"5":0.0547,"6":0.0404}},"unit_mismatch":{"type":"noul","noul":0.3055},"privacy_level":{"type":"score","score":1.2749,"confidence":0.466,"legend":{"0":"very open","1":"open","2":"moderate","3":"cautious","4":"very cautious"},"probabilities":{"0":0.1473,"1":0.466,"2":0.3567,"3":0.0247,"4":0.0054}}},"usage":{"input_tokens":733,"output_tokens":0}},"compare":{"fields":19,"fields_equal":7,"answer_values":[{"question_id":"hides_endpoints","type":"noul","oracle":0.0168,"mine":0.0167,"equal":false},{"question_id":"route_audience","type":"choice","oracle":"followers","mine":"followers","equal":true},{"question_id":"marketing","type":"noul","oracle":0.0045,"mine":0.0045,"equal":true},{"question_id":"zone_boundaries","type":"choice","oracle":"3","mine":"3","equal":true},{"question_id":"unit_mismatch","type":"noul","oracle":0.3055,"mine":0.3013,"equal":false},{"question_id":"privacy_level","type":"score","oracle":1.2749,"mine":1.2756,"equal":false}],"answer_values_equal":3,"answers":6,"choices_equal":"2/2","model_equal":true,"usage_equal":true,"max_abs_diff":0.008199999999999985,"diffs":[{"question_id":"hides_endpoints","field":"noul","oracle":0.0168,"mine":0.0167,"abs_diff":9.99999999999994e-05},{"question_id":"zone_boundaries","field":"confidence","oracle":0.6436,"mine":0.6354,"abs_diff":0.008199999999999985},{"question_id":"zone_boundaries","field":"p[3]","oracle":0.6436,"mine":0.6354,"abs_diff":0.008199999999999985},{"question_id":"zone_boundaries","field":"p[4]","oracle":0.2612,"mine":0.2693,"abs_diff":0.008099999999999996},{"question_id":"zone_boundaries","field":"p[5]","oracle":0.0547,"mine":0.0551,"abs_diff":0.0004000000000000045},{"question_id":"zone_boundaries","field":"p[6]","oracle":0.0404,"mine":0.0402,"abs_diff":0.0001999999999999988},{"question_id":"unit_mismatch","field":"noul","oracle":0.3055,"mine":0.3013,"abs_diff":0.0041999999999999815},{"question_id":"privacy_level","field":"score","oracle":1.2749,"mine":1.2756,"abs_diff":0.000700000000000145},{"question_id":"privacy_level","field":"confidence","oracle":0.466,"mine":0.4661,"abs_diff":9.999999999998899e-05},{"question_id":"privacy_level","field":"p[0]","oracle":0.1473,"mine":0.1468,"abs_diff":0.0004999999999999727},{"question_id":"privacy_level","field":"p[1]","oracle":0.466,"mine":0.4661,"abs_diff":9.999999999998899e-05},{"question_id":"privacy_level","field":"p[2]","oracle":0.3567,"mine":0.357,"abs_diff":0.00029999999999996696}]},"trace":{"host_ids_ms":0.9580419864505529,"tokens":733,"decoder":{"calls":12,"S":64,"call_ms":[133.7968329899013,134.10437499987893,134.00150000234134,134.02883399976417,134.33866601553746,134.39362499048002,135.00833400757983,134.88245802000165,133.81050000316463,134.42816599854268,134.38800000585616,133.9954580180347],"total_ms":1611.1767490510829},"head":{"function":"t1024","host_ms":1.0997080244123936,"head_ms":13.129749975632876},"total_ms":1636.5972499770578}},{"id":"ho_j07","arm":"text","source":"heldout_json","tokens":826,"wall_seconds":1.768070334015647,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9968847454494709,"response":{"model":"clef-flash","answers":{"unassigned":{"type":"choice","choice":"VR-1101","confidence":0.9436,"probabilities":{"VR-1101":0.9436,"VR-1102":0.0137,"VR-1103":0.0136,"VR-1104":0.0121,"none":0.017}},"open_high":{"type":"choice","choice":"2","confidence":0.9636,"probabilities":{"0":0.0076,"1":0.0162,"2":0.9636,"3":0.008,"4":0.0047}},"shared_theme":{"type":"choice","choice":"firmware","confidence":0.995,"probabilities":{"firmware":0.995,"parental":0.0015,"mesh":0.002,"billing":0.0014}},"all_closed":{"type":"noul","noul":0.005},"wider_incident":{"type":"score","score":0.5667,"confidence":0.5667,"legend":{"0":"unlikely","1":"likely"},"probabilities":{"0":0.4333,"1":0.5667}}},"usage":{"input_tokens":826,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"unassigned":{"type":"choice","choice":"VR-1101","confidence":0.9451,"probabilities":{"VR-1101":0.9451,"VR-1102":0.0134,"VR-1103":0.0132,"VR-1104":0.0118,"none":0.0164}},"open_high":{"type":"choice","choice":"2","confidence":0.9631,"probabilities":{"0":0.0077,"1":0.0163,"2":0.9631,"3":0.0081,"4":0.0047}},"shared_theme":{"type":"choice","choice":"firmware","confidence":0.995,"probabilities":{"firmware":0.995,"parental":0.0015,"mesh":0.002,"billing":0.0014}},"all_closed":{"type":"noul","noul":0.005},"wider_incident":{"type":"score","score":0.5652,"confidence":0.5652,"legend":{"0":"unlikely","1":"likely"},"probabilities":{"0":0.4348,"1":0.5652}}},"usage":{"input_tokens":826,"output_tokens":0}},"compare":{"fields":22,"fields_equal":7,"answer_values":[{"question_id":"unassigned","type":"choice","oracle":"VR-1101","mine":"VR-1101","equal":true},{"question_id":"open_high","type":"choice","oracle":"2","mine":"2","equal":true},{"question_id":"shared_theme","type":"choice","oracle":"firmware","mine":"firmware","equal":true},{"question_id":"all_closed","type":"noul","oracle":0.005,"mine":0.005,"equal":true},{"question_id":"wider_incident","type":"score","oracle":0.5652,"mine":0.5667,"equal":false}],"answer_values_equal":4,"answers":5,"choices_equal":"3/3","model_equal":true,"usage_equal":true,"max_abs_diff":0.0015000000000000568,"diffs":[{"question_id":"unassigned","field":"confidence","oracle":0.9451,"mine":0.9436,"abs_diff":0.0015000000000000568},{"question_id":"unassigned","field":"p[VR-1101]","oracle":0.9451,"mine":0.9436,"abs_diff":0.0015000000000000568},{"question_id":"unassigned","field":"p[VR-1102]","oracle":0.0134,"mine":0.0137,"abs_diff":0.0002999999999999999},{"question_id":"unassigned","field":"p[VR-1103]","oracle":0.0132,"mine":0.0136,"abs_diff":0.0003999999999999993},{"question_id":"unassigned","field":"p[VR-1104]","oracle":0.0118,"mine":0.0121,"abs_diff":0.0002999999999999999},{"question_id":"unassigned","field":"p[none]","oracle":0.0164,"mine":0.017,"abs_diff":0.0005999999999999998},{"question_id":"open_high","field":"confidence","oracle":0.9631,"mine":0.9636,"abs_diff":0.000500000000000056},{"question_id":"open_high","field":"p[0]","oracle":0.0077,"mine":0.0076,"abs_diff":0.00010000000000000026},{"question_id":"open_high","field":"p[1]","oracle":0.0163,"mine":0.0162,"abs_diff":9.99999999999994e-05},{"question_id":"open_high","field":"p[2]","oracle":0.9631,"mine":0.9636,"abs_diff":0.000500000000000056},{"question_id":"open_high","field":"p[3]","oracle":0.0081,"mine":0.008,"abs_diff":9.99999999999994e-05},{"question_id":"wider_incident","field":"score","oracle":0.5652,"mine":0.5667,"abs_diff":0.0014999999999999458},{"question_id":"wider_incident","field":"confidence","oracle":0.5652,"mine":0.5667,"abs_diff":0.0014999999999999458},{"question_id":"wider_incident","field":"p[0]","oracle":0.4348,"mine":0.4333,"abs_diff":0.0015000000000000013},{"question_id":"wider_incident","field":"p[1]","oracle":0.5652,"mine":0.5667,"abs_diff":0.0014999999999999458}]},"trace":{"host_ids_ms":0.9575420117471367,"tokens":826,"decoder":{"calls":13,"S":64,"call_ms":[133.06237501092255,133.73079200391658,133.75300000188872,134.93374999961816,134.19695899938233,134.63549999869429,134.8503749759402,135.06745899212547,134.211291995598,134.0395000006538,134.44470800459385,133.93599999835715,134.09158398280852],"total_ms":1744.9532939644996},"head":{"function":"t1024","host_ms":1.0931249998975545,"head_ms":10.312250000424683},"total_ms":1767.2966659883969}},{"id":"ho_i02","arm":"g448","source":"heldout_image","tokens":702,"wall_seconds":1.6055232920043636,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9869932535509092,"response":{"model":"clef-flash","answers":{"harlowick_platform":{"type":"choice","choice":"1","confidence":0.9843,"probabilities":{"1":0.9843,"2":0.0057,"3":0.0057,"4":0.0044}},"earliest":{"type":"choice","choice":"glenvarrow","confidence":0.9633,"probabilities":{"glenvarrow":0.9633,"harlowick":0.0135,"tumbrelby":0.0127,"emberdowne":0.0106}},"platform_3":{"type":"choice","choice":"2","confidence":0.9237,"probabilities":{"0":0.0184,"1":0.0297,"2":0.9237,"3":0.0282}},"after_nine":{"type":"noul","noul":0.9692}},"usage":{"input_tokens":702,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"harlowick_platform":{"type":"choice","choice":"1","confidence":0.9842,"probabilities":{"1":0.9842,"2":0.0057,"3":0.0057,"4":0.0044}},"earliest":{"type":"choice","choice":"glenvarrow","confidence":0.963,"probabilities":{"glenvarrow":0.963,"harlowick":0.0137,"tumbrelby":0.0127,"emberdowne":0.0106}},"platform_3":{"type":"choice","choice":"2","confidence":0.9229,"probabilities":{"0":0.0187,"1":0.03,"2":0.9229,"3":0.0284}},"after_nine":{"type":"noul","noul":0.9691}},"usage":{"input_tokens":702,"output_tokens":0}},"compare":{"fields":16,"fields_equal":5,"answer_values":[{"question_id":"harlowick_platform","type":"choice","oracle":"1","mine":"1","equal":true},{"question_id":"earliest","type":"choice","oracle":"glenvarrow","mine":"glenvarrow","equal":true},{"question_id":"platform_3","type":"choice","oracle":"2","mine":"2","equal":true},{"question_id":"after_nine","type":"noul","oracle":0.9691,"mine":0.9692,"equal":false}],"answer_values_equal":3,"answers":4,"choices_equal":"3/3","model_equal":true,"usage_equal":true,"max_abs_diff":0.0007999999999999119,"diffs":[{"question_id":"harlowick_platform","field":"confidence","oracle":0.9842,"mine":0.9843,"abs_diff":9.999999999998899e-05},{"question_id":"harlowick_platform","field":"p[1]","oracle":0.9842,"mine":0.9843,"abs_diff":9.999999999998899e-05},{"question_id":"earliest","field":"confidence","oracle":0.963,"mine":0.9633,"abs_diff":0.000300000000000078},{"question_id":"earliest","field":"p[glenvarrow]","oracle":0.963,"mine":0.9633,"abs_diff":0.000300000000000078},{"question_id":"earliest","field":"p[harlowick]","oracle":0.0137,"mine":0.0135,"abs_diff":0.00020000000000000052},{"question_id":"platform_3","field":"confidence","oracle":0.9229,"mine":0.9237,"abs_diff":0.0007999999999999119},{"question_id":"platform_3","field":"p[0]","oracle":0.0187,"mine":0.0184,"abs_diff":0.00030000000000000165},{"question_id":"platform_3","field":"p[1]","oracle":0.03,"mine":0.0297,"abs_diff":0.0002999999999999982},{"question_id":"platform_3","field":"p[2]","oracle":0.9229,"mine":0.9237,"abs_diff":0.0007999999999999119},{"question_id":"platform_3","field":"p[3]","oracle":0.0284,"mine":0.0282,"abs_diff":0.00020000000000000226},{"question_id":"after_nine","field":"noul","oracle":0.9691,"mine":0.9692,"abs_diff":9.999999999998899e-05}]},"trace":{"tower":{"grid":448,"preprocess_ms":8.741916011786088,"tower_ms":94.00629199808463,"rows":196},"host_ids_ms":0.9956660214811563,"tokens":702,"decoder":{"calls":11,"S":64,"call_ms":[133.78308399114758,133.65958299254999,134.8236670019105,134.19445799081586,134.4096249958966,134.63795799179934,134.82941698748618,134.66999999945983,134.26516600884497,134.00587500655092,134.6461249922868],"total_ms":1477.9249579587486},"head":{"function":"t1024","host_ms":0.9501249878667295,"head_ms":10.227207996649668},"total_ms":1604.7353339963593},"tower_min_row_cos_vs_oracle":0.9999998819327945},{"id":"ho_i05","arm":"g448","source":"heldout_image","tokens":833,"wall_seconds":1.9918297909898683,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9936648467969846,"response":{"model":"clef-flash","answers":{"first_weekday":{"type":"choice","choice":"mon","confidence":0.8738,"probabilities":{"mon":0.8738,"tue":0.0301,"wed":0.0215,"thu":0.0183,"fri":0.0201,"sat":0.0151,"sun":0.0211}},"circled":{"type":"choice","choice":"14","confidence":0.9791,"probabilities":{"7":0.0066,"14":0.9791,"21":0.0087,"28":0.0056}},"month_length":{"type":"choice","choice":"28","confidence":0.971,"probabilities":{"28":0.971,"29":0.0128,"30":0.0086,"31":0.0075}},"shaded_weekend":{"type":"noul","noul":0.0475},"shaded_count":{"type":"score","score":2.0312,"confidence":0.8255,"legend":{"0":"none","1":"one to three","2":"four to six","3":"seven or more"},"probabilities":{"0":0.0344,"1":0.0199,"2":0.8255,"3":0.1201}}},"usage":{"input_tokens":833,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"first_weekday":{"type":"choice","choice":"mon","confidence":0.8736,"probabilities":{"mon":0.8736,"tue":0.0302,"wed":0.0215,"thu":0.0183,"fri":0.0201,"sat":0.0151,"sun":0.0211}},"circled":{"type":"choice","choice":"14","confidence":0.9791,"probabilities":{"7":0.0066,"14":0.9791,"21":0.0087,"28":0.0056}},"month_length":{"type":"choice","choice":"28","confidence":0.9712,"probabilities":{"28":0.9712,"29":0.0128,"30":0.0086,"31":0.0074}},"shaded_weekend":{"type":"noul","noul":0.0479},"shaded_count":{"type":"score","score":2.0317,"confidence":0.8242,"legend":{"0":"none","1":"one to three","2":"four to six","3":"seven or more"},"probabilities":{"0":0.0346,"1":0.0201,"2":0.8242,"3":0.1211}}},"usage":{"input_tokens":833,"output_tokens":0}},"compare":{"fields":25,"fields_equal":12,"answer_values":[{"question_id":"first_weekday","type":"choice","oracle":"mon","mine":"mon","equal":true},{"question_id":"circled","type":"choice","oracle":"14","mine":"14","equal":true},{"question_id":"month_length","type":"choice","oracle":"28","mine":"28","equal":true},{"question_id":"shaded_weekend","type":"noul","oracle":0.0479,"mine":0.0475,"equal":false},{"question_id":"shaded_count","type":"score","oracle":2.0317,"mine":2.0312,"equal":false}],"answer_values_equal":3,"answers":5,"choices_equal":"3/3","model_equal":true,"usage_equal":true,"max_abs_diff":0.0012999999999999678,"diffs":[{"question_id":"first_weekday","field":"confidence","oracle":0.8736,"mine":0.8738,"abs_diff":0.00019999999999997797},{"question_id":"first_weekday","field":"p[mon]","oracle":0.8736,"mine":0.8738,"abs_diff":0.00019999999999997797},{"question_id":"first_weekday","field":"p[tue]","oracle":0.0302,"mine":0.0301,"abs_diff":0.00010000000000000286},{"question_id":"month_length","field":"confidence","oracle":0.9712,"mine":0.971,"abs_diff":0.00019999999999997797},{"question_id":"month_length","field":"p[28]","oracle":0.9712,"mine":0.971,"abs_diff":0.00019999999999997797},{"question_id":"month_length","field":"p[31]","oracle":0.0074,"mine":0.0075,"abs_diff":9.99999999999994e-05},{"question_id":"shaded_weekend","field":"noul","oracle":0.0479,"mine":0.0475,"abs_diff":0.0003999999999999976},{"question_id":"shaded_count","field":"score","oracle":2.0317,"mine":2.0312,"abs_diff":0.0004999999999997229},{"question_id":"shaded_count","field":"confidence","oracle":0.8242,"mine":0.8255,"abs_diff":0.0012999999999999678},{"question_id":"shaded_count","field":"p[0]","oracle":0.0346,"mine":0.0344,"abs_diff":0.0001999999999999988},{"question_id":"shaded_count","field":"p[1]","oracle":0.0201,"mine":0.0199,"abs_diff":0.0001999999999999988},{"question_id":"shaded_count","field":"p[2]","oracle":0.8242,"mine":0.8255,"abs_diff":0.0012999999999999678},{"question_id":"shaded_count","field":"p[3]","oracle":0.1211,"mine":0.1201,"abs_diff":0.0010000000000000009}]},"trace":{"tower":{"grid":448,"preprocess_ms":5.228083988185972,"tower_ms":80.43670799816027,"rows":196},"host_ids_ms":1.0556670022197068,"tokens":833,"decoder":{"calls":14,"S":64,"call_ms":[133.60562501475215,134.30787497782148,134.050708002178,134.320375014795,134.5825840253383,134.887374995742,134.78020901675336,134.89316700724885,133.81412500166334,134.09658300224692,134.27191600203514,134.30754200089723,134.21104100416414,134.46729100542143],"total_ms":1880.5964160710573},"head":{"function":"t1024","host_ms":1.1997090186923742,"head_ms":10.28470799792558},"total_ms":1991.0552090150304},"tower_min_row_cos_vs_oracle":0.9999999963921723},{"id":"ho_p01","arm":"g448","source":"heldout_photo","tokens":766,"wall_seconds":1.7303035000222735,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.986856741138263,"response":{"model":"clef-flash","answers":{"lamp_on":{"type":"noul","noul":0.9472},"time_of_day":{"type":"choice","choice":"twilight","confidence":0.9618,"probabilities":{"midday":0.007,"twilight":0.9618,"night":0.0249,"grey_day":0.0063}},"sea_behind":{"type":"noul","noul":0.9286},"tower_colour":{"type":"choice","choice":"white","confidence":0.9487,"probabilities":{"white":0.9487,"red_white":0.0245,"black":0.0133,"yellow":0.0135}},"remoteness":{"type":"score","score":1.8457,"confidence":0.8852,"legend":{"0":"in a busy town","1":"at the edge of a village","2":"isolated"},"probabilities":{"0":0.0394,"1":0.0754,"2":0.8852}}},"usage":{"input_tokens":766,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"lamp_on":{"type":"noul","noul":0.9475},"time_of_day":{"type":"choice","choice":"twilight","confidence":0.9617,"probabilities":{"midday":0.007,"twilight":0.9617,"night":0.025,"grey_day":0.0063}},"sea_behind":{"type":"noul","noul":0.9287},"tower_colour":{"type":"choice","choice":"white","confidence":0.9487,"probabilities":{"white":0.9487,"red_white":0.0245,"black":0.0133,"yellow":0.0135}},"remoteness":{"type":"score","score":1.8459,"confidence":0.8853,"legend":{"0":"in a busy town","1":"at the edge of a village","2":"isolated"},"probabilities":{"0":0.0395,"1":0.0752,"2":0.8853}}},"usage":{"input_tokens":766,"output_tokens":0}},"compare":{"fields":17,"fields_equal":7,"answer_values":[{"question_id":"lamp_on","type":"noul","oracle":0.9475,"mine":0.9472,"equal":false},{"question_id":"time_of_day","type":"choice","oracle":"twilight","mine":"twilight","equal":true},{"question_id":"sea_behind","type":"noul","oracle":0.9287,"mine":0.9286,"equal":false},{"question_id":"tower_colour","type":"choice","oracle":"white","mine":"white","equal":true},{"question_id":"remoteness","type":"score","oracle":1.8459,"mine":1.8457,"equal":false}],"answer_values_equal":2,"answers":5,"choices_equal":"2/2","model_equal":true,"usage_equal":true,"max_abs_diff":0.00029999999999996696,"diffs":[{"question_id":"lamp_on","field":"noul","oracle":0.9475,"mine":0.9472,"abs_diff":0.00029999999999996696},{"question_id":"time_of_day","field":"confidence","oracle":0.9617,"mine":0.9618,"abs_diff":9.999999999998899e-05},{"question_id":"time_of_day","field":"p[twilight]","oracle":0.9617,"mine":0.9618,"abs_diff":9.999999999998899e-05},{"question_id":"time_of_day","field":"p[night]","oracle":0.025,"mine":0.0249,"abs_diff":0.00010000000000000286},{"question_id":"sea_behind","field":"noul","oracle":0.9287,"mine":0.9286,"abs_diff":9.999999999998899e-05},{"question_id":"remoteness","field":"score","oracle":1.8459,"mine":1.8457,"abs_diff":0.00020000000000020002},{"question_id":"remoteness","field":"confidence","oracle":0.8853,"mine":0.8852,"abs_diff":9.999999999998899e-05},{"question_id":"remoteness","field":"p[0]","oracle":0.0395,"mine":0.0394,"abs_diff":0.00010000000000000286},{"question_id":"remoteness","field":"p[1]","oracle":0.0752,"mine":0.0754,"abs_diff":0.00019999999999999185},{"question_id":"remoteness","field":"p[2]","oracle":0.8853,"mine":0.8852,"abs_diff":9.999999999998899e-05}]},"trace":{"tower":{"grid":448,"preprocess_ms":14.9066669982858,"tower_ms":80.18166699912399,"rows":196},"host_ids_ms":0.9102910116780549,"tokens":766,"decoder":{"calls":12,"S":64,"call_ms":[133.50987501326017,133.77520799986087,134.53608300187625,134.79449998703785,134.04495801660232,134.5472919929307,135.13650000095367,135.1164579973556,133.96554201608524,133.8553329987917,134.23070797580294,134.2282090045046],"total_ms":1611.740666005062},"head":{"function":"t1024","host_ms":0.9424170129932463,"head_ms":9.955207991879433},"total_ms":1729.5421250164509},"tower_min_row_cos_vs_oracle":0.9999999974837644},{"id":"ho_i01","arm":"g256","source":"heldout_image","tokens":529,"wall_seconds":1.283632333012065,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9980640097138683,"response":{"model":"clef-flash","answers":{"biggest_slice":{"type":"choice","choice":"orange","confidence":0.9584,"probabilities":{"orange":0.9584,"blue":0.0317,"grey":0.0057,"yellow":0.0042}},"over_half":{"type":"noul","noul":0.266},"smallest_label":{"type":"choice","choice":"light","confidence":0.9777,"probabilities":{"heat":0.0046,"water":0.0079,"other":0.0098,"light":0.9777}},"blue_share":{"type":"score","score":0.9977,"confidence":0.9486,"legend":{"0":"under a quarter","1":"between a quarter and a half","2":"over a half"},"probabilities":{"0":0.0268,"1":0.9486,"2":0.0246}}},"usage":{"input_tokens":529,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"biggest_slice":{"type":"choice","choice":"orange","confidence":0.9582,"probabilities":{"orange":0.9582,"blue":0.0318,"grey":0.0057,"yellow":0.0042}},"over_half":{"type":"noul","noul":0.2644},"smallest_label":{"type":"choice","choice":"light","confidence":0.9775,"probabilities":{"heat":0.0046,"water":0.008,"other":0.0099,"light":0.9775}},"blue_share":{"type":"score","score":0.9979,"confidence":0.9486,"legend":{"0":"under a quarter","1":"between a quarter and a half","2":"over a half"},"probabilities":{"0":0.0268,"1":0.9486,"2":0.0246}}},"usage":{"input_tokens":529,"output_tokens":0}},"compare":{"fields":16,"fields_equal":7,"answer_values":[{"question_id":"biggest_slice","type":"choice","oracle":"orange","mine":"orange","equal":true},{"question_id":"over_half","type":"noul","oracle":0.2644,"mine":0.266,"equal":false},{"question_id":"smallest_label","type":"choice","oracle":"light","mine":"light","equal":true},{"question_id":"blue_share","type":"score","oracle":0.9979,"mine":0.9977,"equal":false}],"answer_values_equal":2,"answers":4,"choices_equal":"2/2","model_equal":true,"usage_equal":true,"max_abs_diff":0.0015999999999999903,"diffs":[{"question_id":"biggest_slice","field":"confidence","oracle":0.9582,"mine":0.9584,"abs_diff":0.00019999999999997797},{"question_id":"biggest_slice","field":"p[orange]","oracle":0.9582,"mine":0.9584,"abs_diff":0.00019999999999997797},{"question_id":"biggest_slice","field":"p[blue]","oracle":0.0318,"mine":0.0317,"abs_diff":0.00010000000000000286},{"question_id":"over_half","field":"noul","oracle":0.2644,"mine":0.266,"abs_diff":0.0015999999999999903},{"question_id":"smallest_label","field":"confidence","oracle":0.9775,"mine":0.9777,"abs_diff":0.00019999999999997797},{"question_id":"smallest_label","field":"p[water]","oracle":0.008,"mine":0.0079,"abs_diff":9.99999999999994e-05},{"question_id":"smallest_label","field":"p[other]","oracle":0.0099,"mine":0.0098,"abs_diff":0.00010000000000000113},{"question_id":"smallest_label","field":"p[light]","oracle":0.9775,"mine":0.9777,"abs_diff":0.00019999999999997797},{"question_id":"blue_share","field":"score","oracle":0.9979,"mine":0.9977,"abs_diff":0.00019999999999997797}]},"trace":{"tower":{"grid":256,"preprocess_ms":3.38866698439233,"tower_ms":47.42291700677015,"rows":64},"host_ids_ms":0.7696670072618872,"tokens":529,"decoder":{"calls":9,"S":64,"call_ms":[134.591707988875,133.97495800745673,134.21945800655521,134.6251659851987,134.025917010149,134.52891699853353,135.02970797708258,134.597666998161,134.02358299936168],"total_ms":1209.6170819713734},"head":{"function":"t1024","host_ms":0.8073749777395278,"head_ms":9.749916003784165},"total_ms":1282.8498329909053},"tower_min_row_cos_vs_oracle":0.9999999997192758},{"id":"ho_p02","arm":"g256","source":"heldout_photo","tokens":620,"wall_seconds":1.4205946250003763,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.997943696018564,"response":{"model":"clef-flash","answers":{"petal_colour":{"type":"choice","choice":"yellow","confidence":0.9689,"probabilities":{"yellow":0.9689,"white":0.0098,"purple":0.0105,"red":0.0108}},"insect":{"type":"noul","noul":0.0311},"centre_colour":{"type":"choice","choice":"brown","confidence":0.9665,"probabilities":{"brown":0.9665,"green":0.014,"blue":0.0088,"white":0.0108}},"setting":{"type":"choice","choice":"outdoors","confidence":0.9742,"probabilities":{"vase":0.0104,"outdoors":0.9742,"sand":0.0076,"snow":0.0077}},"density":{"type":"score","score":1.585,"confidence":0.6525,"legend":{"0":"sparse","1":"moderate","2":"dense"},"probabilities":{"0":0.0675,"1":0.2801,"2":0.6525}}},"usage":{"input_tokens":620,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"petal_colour":{"type":"choice","choice":"yellow","confidence":0.9724,"probabilities":{"yellow":0.9724,"white":0.0087,"purple":0.0094,"red":0.0095}},"insect":{"type":"noul","noul":0.0314},"centre_colour":{"type":"choice","choice":"brown","confidence":0.9677,"probabilities":{"brown":0.9677,"green":0.0136,"blue":0.0084,"white":0.0103}},"setting":{"type":"choice","choice":"outdoors","confidence":0.975,"probabilities":{"vase":0.0101,"outdoors":0.975,"sand":0.0074,"snow":0.0075}},"density":{"type":"score","score":1.5856,"confidence":0.6526,"legend":{"0":"sparse","1":"moderate","2":"dense"},"probabilities":{"0":0.067,"1":0.2804,"2":0.6526}}},"usage":{"input_tokens":620,"output_tokens":0}},"compare":{"fields":21,"fields_equal":0,"answer_values":[{"question_id":"petal_colour","type":"choice","oracle":"yellow","mine":"yellow","equal":true},{"question_id":"insect","type":"noul","oracle":0.0314,"mine":0.0311,"equal":false},{"question_id":"centre_colour","type":"choice","oracle":"brown","mine":"brown","equal":true},{"question_id":"setting","type":"choice","oracle":"outdoors","mine":"outdoors","equal":true},{"question_id":"density","type":"score","oracle":1.5856,"mine":1.585,"equal":false}],"answer_values_equal":3,"answers":5,"choices_equal":"3/3","model_equal":true,"usage_equal":true,"max_abs_diff":0.0035000000000000586,"diffs":[{"question_id":"petal_colour","field":"confidence","oracle":0.9724,"mine":0.9689,"abs_diff":0.0035000000000000586},{"question_id":"petal_colour","field":"p[yellow]","oracle":0.9724,"mine":0.9689,"abs_diff":0.0035000000000000586},{"question_id":"petal_colour","field":"p[white]","oracle":0.0087,"mine":0.0098,"abs_diff":0.0011000000000000003},{"question_id":"petal_colour","field":"p[purple]","oracle":0.0094,"mine":0.0105,"abs_diff":0.0011000000000000003},{"question_id":"petal_colour","field":"p[red]","oracle":0.0095,"mine":0.0108,"abs_diff":0.0013000000000000008},{"question_id":"insect","field":"noul","oracle":0.0314,"mine":0.0311,"abs_diff":0.0002999999999999982},{"question_id":"centre_colour","field":"confidence","oracle":0.9677,"mine":0.9665,"abs_diff":0.0011999999999999789},{"question_id":"centre_colour","field":"p[brown]","oracle":0.9677,"mine":0.9665,"abs_diff":0.0011999999999999789},{"question_id":"centre_colour","field":"p[green]","oracle":0.0136,"mine":0.014,"abs_diff":0.00040000000000000105},{"question_id":"centre_colour","field":"p[blue]","oracle":0.0084,"mine":0.0088,"abs_diff":0.00040000000000000105},{"question_id":"centre_colour","field":"p[white]","oracle":0.0103,"mine":0.0108,"abs_diff":0.0005000000000000004},{"question_id":"setting","field":"confidence","oracle":0.975,"mine":0.9742,"abs_diff":0.0008000000000000229},{"question_id":"setting","field":"p[vase]","oracle":0.0101,"mine":0.0104,"abs_diff":0.0002999999999999999},{"question_id":"setting","field":"p[outdoors]","oracle":0.975,"mine":0.9742,"abs_diff":0.0008000000000000229},{"question_id":"setting","field":"p[sand]","oracle":0.0074,"mine":0.0076,"abs_diff":0.00019999999999999966},{"question_id":"setting","field":"p[snow]","oracle":0.0075,"mine":0.0077,"abs_diff":0.00020000000000000052},{"question_id":"density","field":"score","oracle":1.5856,"mine":1.585,"abs_diff":0.0005999999999999339},{"question_id":"density","field":"confidence","oracle":0.6526,"mine":0.6525,"abs_diff":9.999999999998899e-05},{"question_id":"density","field":"p[0]","oracle":0.067,"mine":0.0675,"abs_diff":0.0005000000000000004},{"question_id":"density","field":"p[1]","oracle":0.2804,"mine":0.2801,"abs_diff":0.00029999999999996696},{"question_id":"density","field":"p[2]","oracle":0.6526,"mine":0.6525,"abs_diff":9.999999999998899e-05}]},"trace":{"tower":{"grid":256,"preprocess_ms":20.864791003987193,"tower_ms":32.58545900462195,"rows":64},"host_ids_ms":0.8701249898876995,"tokens":620,"decoder":{"calls":10,"S":64,"call_ms":[133.53995900251903,133.7557919905521,133.76541700563394,134.43991701933555,134.617875010008,134.60366599611007,135.13116599642672,134.94599997648038,134.3041660147719,134.50275000650436],"total_ms":1343.606708018342},"head":{"function":"t1024","host_ms":0.9919580188579857,"head_ms":10.205999977188185},"total_ms":1419.7891670046374},"tower_min_row_cos_vs_oracle":0.9999999988067613}]}},"export":{"bucket":{"name":"clef_flash_head_bucket_fp16w32","argv":["--shape","bucket","--weights","fp16w32","--aot","--record","<lane>/logs/r5_export_head_bucket.json"],"pid":85244,"started":"2026-10-03T13:53:16+09:00","script_sha256":"8af91937bf1869d8c39aa5b05b2e1799297d4f36013a5793753183bdb6a2e930","clef_head_sha256":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d","export":{"weights":"fp16w32","shape":"bucket","buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"op_census":{"aten_ops":{"aten._assert_tensor_metadata.default":122,"aten.abs.default":1,"aten.add.Tensor":35,"aten.cat.default":1,"aten.clamp.default":2,"aten.clamp_min.default":4,"aten.div.Tensor":4,"aten.embedding.default":1,"aten.exp.default":2,"aten.gelu.default":7,"aten.layer_norm.default":22,"aten.linalg_vector_norm.default":4,"aten.linear.default":60,"aten.matmul.default":28,"aten.mul.Tensor":21,"aten.reshape.default":40,"aten.sigmoid.default":1,"aten.slice.Tensor":60,"aten.softmax.int":11,"aten.squeeze.dim":1,"aten.sub.Tensor":4,"aten.sum.dim_IntList":2,"aten.to.dtype":122,"aten.transpose.int":52},"after_coreai_decomposition":{"<built-in function getitem>":22,"aten._assert_tensor_metadata.default":122,"aten._softmax.default":11,"aten._to_copy.default":122,"aten.abs.default":1,"aten.add.Tensor":35,"aten.addmm.default":54,"aten.bmm.default":20,"aten.cat.default":1,"aten.clamp.default":6,"aten.clone.default":10,"aten.div.Tensor":4,"aten.embedding.default":1,"aten.exp.default":2,"aten.expand.default":40,"aten.gelu.default":7,"aten.linalg_vector_norm.default":4,"aten.mm.default":14,"aten.mul.Tensor":21,"aten.native_layer_norm.default":22,"aten.permute.default":112,"aten.sigmoid.default":1,"aten.slice.Tensor":60,"aten.squeeze.dims":1,"aten.sub.Tensor":4,"aten.sum.dim_IntList":2,"aten.view.default":100},"data_dependent_ops":[],"exported_vs_eager_max_abs":0.0,"range_constraints":{}},"export_attempts":[{"externalize":"default externalize specs","ok":true,"seconds":9.44808970799204}],"functions":["t512","t1024","t2048","t4096"],"aimodel":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.aimodel","aimodel_files":{"main.hash":32,"main.mlirb":243978965,"metadata.json":105},"aimodel_bytes":243979102,"main_mlirb":{"bytes":243978965,"sha256":"51e512ad1e4d3bcfdc7af43c1100124bd94d539de01f09651afc9b98486856fe"},"seconds":{"build":0.19440945898531936,"export":10.82508116599638,"optimize":0.031602208997355774,"save":0.2501100410008803,"total":11.301202874979936},"aot":{"aimodelc":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c"],"seconds":0.5527557500172406,"coreai_build":"/var/run/com.apple.security.cryptexd/mnt/com.apple.MobileAsset.MetalToolchain-v27.1.266.1.6t2kcX/Metal.xctoolchain/usr/bin/coreai-build","command":["/var/run/com.apple.security.cryptexd/mnt/com.apple.MobileAsset.MetalToolchain-v27.1.266.1.6t2kcX/Metal.xctoolchain/usr/bin/coreai-build","compile","<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.aimodel","--output","<lane>/exports/head/clef_flash_head_bucket_fp16w32","--platform","macOS","--preferred-compute","gpu","--architecture","h16c"],"digest":{"bytes":244402049,"tree_sha256":"a2a61b0b2a05962f8c20cbe4235f415484c1f48a0ebdaee930f99048777fd4f4","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"2453d7beeba64a7ecd5b4bcbe87ff3b70bf30c95bca8495a61777f3179403621","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"4d7901f065a3073dfa2b8733078eb0f2052a156a1a79d1b6b53e78bdf3461566","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_0.mpsgraph":"52f24f36572ecf9cb0e991b23b6be03a20042a92edb3c6cbdcc7e86ac8867668","main-h16c.mlirb":"2067eb429a27b78a2e0668eacf8b5d67573f487cf237feac90627555e79c79c4","main.hash":"31013f3df25a93cdc4f35f00c4126117ba1945fcfc703b9210b197247ca024f2","metadata.json":"143e11a32f759ad837ea43753e71e125e6783ee3dd647b91bf16dc4cc36ee7d5","stats.json":"ec95a2ef6d93a552887a0a816a0f27fe728abda36bf96b50727a5290a6b8c21c"}}}},"metadata_sha256":"631219aafcdb56ef1f008e5476f0e7a282933abdc51b4835132676d97124ab20","finished":"2026-10-03T13:53:31+09:00"},"dynamic":{"name":"clef_flash_head_dynamic_fp16w32","argv":["--shape","dynamic","--weights","fp16w32","--q-min","1","--aot","--record","<lane>/logs/r5_export_head_dynamic.json"],"pid":76978,"started":"2026-10-03T13:45:49+09:00","script_sha256":"866a1451f9d91a391813db14f7b8bb70b16ebb71422e7ca9ab67a592285a7dcf","clef_head_sha256":"4b2104a83b83b6d124d45c485be829f893a8b2b5bc4c6a76545e743080dadda7","export":{"weights":"fp16w32","shape":"dynamic","dims":{"T":[64,4096],"Q":[1,16],"O":[2,128]},"example":{"run":"own_t04/text","T":512,"Q":4,"O":11},"op_census":{"aten_ops":{"aten._assert_tensor_metadata.default":122,"aten.abs.default":1,"aten.add.Tensor":35,"aten.cat.default":1,"aten.clamp.default":2,"aten.clamp_min.default":4,"aten.div.Tensor":4,"aten.embedding.default":1,"aten.exp.default":2,"aten.gelu.default":7,"aten.layer_norm.default":22,"aten.linalg_vector_norm.default":4,"aten.linear.default":60,"aten.matmul.default":28,"aten.mul.Tensor":21,"aten.reshape.default":40,"aten.sigmoid.default":1,"aten.slice.Tensor":60,"aten.softmax.int":11,"aten.squeeze.dim":1,"aten.sub.Tensor":4,"aten.sum.dim_IntList":2,"aten.sym_size.int":3,"aten.to.dtype":122,"aten.transpose.int":52},"after_coreai_decomposition":{"<built-in function getitem>":22,"aten._assert_tensor_metadata.default":122,"aten._softmax.default":11,"aten._to_copy.default":122,"aten.abs.default":1,"aten.add.Tensor":35,"aten.addmm.default":54,"aten.bmm.default":20,"aten.cat.default":1,"aten.clamp.default":6,"aten.clone.default":10,"aten.div.Tensor":4,"aten.embedding.default":1,"aten.exp.default":2,"aten.expand.default":40,"aten.gelu.default":7,"aten.linalg_vector_norm.default":4,"aten.mm.default":14,"aten.mul.Tensor":21,"aten.native_layer_norm.default":22,"aten.permute.default":112,"aten.sigmoid.default":1,"aten.slice.Tensor":60,"aten.squeeze.dims":1,"aten.sub.Tensor":4,"aten.sum.dim_IntList":2,"aten.sym_size.int":3,"aten.view.default":100},"data_dependent_ops":[],"exported_vs_eager_max_abs":0.0,"range_constraints":{"s19":"VR[64, 4096]","s40":"VR[1, 16]","s12":"VR[2, 128]"}},"export_attempts":[{"externalize":"default externalize specs","ok":true,"seconds":3.40813049999997}],"functions":["main"],"aimodel":"<lane>/exports/head/clef_flash_head_dynamic_fp16w32/clef_flash_head_dynamic_fp16w32.aimodel","aimodel_files":{"main.hash":32,"main.mlirb":243814609,"metadata.json":105},"aimodel_bytes":243814746,"main_mlirb":{"bytes":243814609,"sha256":"77cfe978c8c62e85dbf5f28f1052563d404c7f26adcbcb8732c6db2a6bdc9725"},"seconds":{"build":0.18020279199117795,"export":5.378591707994929,"optimize":0.01343729201471433,"save":0.14021599999978207,"total":5.712447792000603},"aot":{"aimodelc":"<lane>/exports/head/clef_flash_head_dynamic_fp16w32/clef_flash_head_dynamic_fp16w32.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":1.7011619999830145,"coreai_build":"/var/run/com.apple.security.cryptexd/mnt/com.apple.MobileAsset.MetalToolchain-v27.1.266.1.6t2kcX/Metal.xctoolchain/usr/bin/coreai-build","command":["/var/run/com.apple.security.cryptexd/mnt/com.apple.MobileAsset.MetalToolchain-v27.1.266.1.6t2kcX/Metal.xctoolchain/usr/bin/coreai-build","compile","<lane>/exports/head/clef_flash_head_dynamic_fp16w32/clef_flash_head_dynamic_fp16w32.aimodel","--output","<lane>/exports/head/clef_flash_head_dynamic_fp16w32","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"digest":{"bytes":244135643,"tree_sha256":"27d65a688d023609320f821c06f7ad8f33f9a11175ef1b91f9bb91fb717b653f","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"2b24864e952a5723517335bc713fdd5be5e95c2dc90b023ba3d2c42be6d07f1c","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"7dc85823af673476ff6f9644dea4e669773e151ce754d48234752b4fe92067b2","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"5509ee7fd98a6722507d28fabac97579d5b12c975480be8ada1a8c67805e0a36","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"0a080fa9ca923eff3ea55461db369d15c7216c9ff313ba8119639b9692d3a385","main-h16c.mlirb":"dd09b963384c4f9d977f6683e6a8c09f639ba12228a73191108e5b3f3f52eae6","main.hash":"f6003fb47bdd07d4dded423e87e34807ca4bd6b949333977c1d6b331ab7dbdfe","metadata.json":"d92a631c4445c94ac769e5fc5d3d50b833e3237f4816a9e59b60f5cc8e8f860a","stats.json":"43c36b27d7f430775da0c01a06027d292c770fc6a110972d85c11cb4a6b0024d"}}}},"metadata_sha256":"a77009dbfa91bd2a84c0cc11bfd30d04d84c1ff82edc51f6a13ffc6eb884593b","finished":"2026-10-03T13:45:58+09:00"},"dynamic_first_call":{"what":"the dynamic-shape AOT (built with --expect-frequent-reshapes) loads and aborts at its first call, at the traced shape too; logs verbatim","h1_run_log":["2026-10-03 13:46:44 start: <coreai-models venv>/bin/python gate_head.py h1 --head <lane>/exports/head/clef_flash_head_dynamic_fp16w32 --tag head_h1_dynamic --out <lane>/results/head_h1.json","shard_00: 36 runs + re-run","/AppleInternal/Library/BuildRoots/4~CV2iugCS017GRPdEjj32gfBJXr8YaxAcCWYlUuw/Library/Caches/com.apple.xbs/TemporaryDirectory.WK6jPd/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Runtimes/MPSRuntime/Operations/GPUMemrefOps.mm:164: failed assertion `Failed to resolve dynamic dimensions for memref.alloc'","shard_00: worker failed (exit -6)","2026-10-03 13:46:45 exit 1"],"probe":[{"T":512,"Q":4,"O":11,"stage":"loaded","load_s":0.0045458330132532865},{"T":512,"Q":4,"O":11,"stage":"function","load_s":0.0045458330132532865,"inputs":{"hidden":[[-1,4096],"float32"],"key_valid":[[-1],"float32"],"q_avg":[[-1,-1],"float32"],"o_avg":[[-1,-1],"float32"],"g_avg":[[1,-1],"float32"],"lexical":[[-1,4096],"float32"],"member":[[-1,-1],"float32"],"type_ids":[[-1],"int32"],"q_valid":[[-1],"float32"],"o_valid":[[-1],"float32"]}},"/AppleInternal/Library/BuildRoots/4~CV2iugCS017GRPdEjj32gfBJXr8YaxAcCWYlUuw/Library/Caches/com.apple.xbs/TemporaryDirectory.WK6jPd/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Runtimes/MPSRuntime/Operations/GPUMemrefOps.mm:164: failed assertion `Failed to resolve dynamic dimensions for memref.alloc'","T=512 Q=4 O=11 exit 134",{"T":448,"Q":3,"O":10,"stage":"loaded","load_s":0.0044535000051837415},{"T":448,"Q":3,"O":10,"stage":"function","load_s":0.0044535000051837415,"inputs":{"hidden":[[-1,4096],"float32"],"key_valid":[[-1],"float32"],"q_avg":[[-1,-1],"float32"],"o_avg":[[-1,-1],"float32"],"g_avg":[[1,-1],"float32"],"lexical":[[-1,4096],"float32"],"member":[[-1,-1],"float32"],"type_ids":[[-1],"int32"],"q_valid":[[-1],"float32"],"o_valid":[[-1],"float32"]}},"/AppleInternal/Library/BuildRoots/4~CV2iugCS017GRPdEjj32gfBJXr8YaxAcCWYlUuw/Library/Caches/com.apple.xbs/TemporaryDirectory.WK6jPd/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Runtimes/MPSRuntime/Operations/GPUMemrefOps.mm:164: failed assertion `Failed to resolve dynamic dimensions for memref.alloc'","T=448 Q=3 O=10 exit 134"]},"dynamic_without_efr":{"what":"the same dynamic export AOT-compiled without --expect-frequent-reshapes: runs and matches the eager head; every new input shape costs a first call of 0.27-1.5 s in a process (shapes A B A B C A B, ms per call)","probe":[[["A",[512,4,11],290.3],["B",[448,3,10],272.4],["A",[512,4,11],8.3],["B",[448,3,10],4.8],["C",[1024,2,6],1514.6],["A",[512,4,11],9.5],["B",[448,3,10],6.0]],{"dynamic_no_efr_vs_eager":"own_t01/text","shape":[448,3,10],"max_abs_dlogit":7.152557373046875e-07},{"dynamic_no_efr_vs_eager":"own_t15/text","shape":[2624,6,22],"max_abs_dlogit":9.5367431640625e-07},{"dynamic_no_efr_vs_eager":"semif_a3f18f3a63d45345942b/text","shape":[192,1,3],"max_abs_dlogit":4.76837158203125e-07}]},"op_census":{"t512":{"aten_ops":{"aten._assert_tensor_metadata.default":122,"aten.abs.default":1,"aten.add.Tensor":35,"aten.cat.default":1,"aten.clamp.default":2,"aten.clamp_min.default":4,"aten.div.Tensor":4,"aten.embedding.default":1,"aten.exp.default":2,"aten.gelu.default":7,"aten.layer_norm.default":22,"aten.linalg_vector_norm.default":4,"aten.linear.default":60,"aten.matmul.default":28,"aten.mul.Tensor":21,"aten.reshape.default":40,"aten.sigmoid.default":1,"aten.slice.Tensor":60,"aten.softmax.int":11,"aten.squeeze.dim":1,"aten.sub.Tensor":4,"aten.sum.dim_IntList":2,"aten.to.dtype":122,"aten.transpose.int":52},"after_coreai_decomposition":{"<built-in function getitem>":22,"aten._assert_tensor_metadata.default":122,"aten._softmax.default":11,"aten._to_copy.default":122,"aten.abs.default":1,"aten.add.Tensor":35,"aten.addmm.default":54,"aten.bmm.default":20,"aten.cat.default":1,"aten.clamp.default":6,"aten.clone.default":10,"aten.div.Tensor":4,"aten.embedding.default":1,"aten.exp.default":2,"aten.expand.default":40,"aten.gelu.default":7,"aten.linalg_vector_norm.default":4,"aten.mm.default":14,"aten.mul.Tensor":21,"aten.native_layer_norm.default":22,"aten.permute.default":112,"aten.sigmoid.default":1,"aten.slice.Tensor":60,"aten.squeeze.dims":1,"aten.sub.Tensor":4,"aten.sum.dim_IntList":2,"aten.view.default":100},"exported_vs_eager_max_abs":0.0,"data_dependent_ops":[]},"t4096":{"aten_ops":{"aten._assert_tensor_metadata.default":122,"aten.abs.default":1,"aten.add.Tensor":53,"aten.amax.default":12,"aten.cat.default":1,"aten.clamp.default":2,"aten.clamp_min.default":4,"aten.div.Tensor":10,"aten.embedding.default":1,"aten.exp.default":14,"aten.gelu.default":7,"aten.layer_norm.default":22,"aten.linalg_vector_norm.default":4,"aten.linear.default":60,"aten.matmul.default":40,"aten.maximum.default":6,"aten.mul.Tensor":27,"aten.reshape.default":40,"aten.sigmoid.default":1,"aten.slice.Tensor":96,"aten.softmax.int":5,"aten.squeeze.dim":1,"aten.sub.Tensor":16,"aten.sum.dim_IntList":14,"aten.to.dtype":122,"aten.transpose.int":58},"after_coreai_decomposition":{"<built-in function getitem>":22,"aten._assert_tensor_metadata.default":122,"aten._softmax.default":5,"aten._to_copy.default":122,"aten.abs.default":1,"aten.add.Tensor":53,"aten.addmm.default":54,"aten.amax.default":12,"aten.bmm.default":32,"aten.cat.default":1,"aten.clamp.default":6,"aten.clone.default":10,"aten.div.Tensor":10,"aten.embedding.default":1,"aten.exp.default":14,"aten.expand.default":64,"aten.gelu.default":7,"aten.linalg_vector_norm.default":4,"aten.maximum.default":6,"aten.mm.default":14,"aten.mul.Tensor":27,"aten.native_layer_norm.default":22,"aten.permute.default":118,"aten.sigmoid.default":1,"aten.slice.Tensor":96,"aten.squeeze.dims":1,"aten.sub.Tensor":16,"aten.sum.dim_IntList":14,"aten.view.default":136},"exported_vs_eager_max_abs":0.0,"data_dependent_ops":[]},"ops_only_in_t4096":["aten.amax.default","aten.maximum.default"]},"author_head_export_scout":{"torch":"2.9.0","shape":{"T":512,"Q":4,"O":12,"records":1},"config":{"hidden_size":4096,"width":1024,"routing_layers":2,"layers":4,"heads":16,"feedforward":4096},"state_dict":[{"key":"prior_logit_scale","shape":[]},{"key":"joint_logit_scale","shape":[]},{"key":"residual_gate","shape":[]},{"key":"hidden_norm.weight","shape":[4096]},{"key":"hidden_norm.bias","shape":[4096]},{"key":"memory_projection.weight","shape":[1024,4096]},{"key":"question_projection.weight","shape":[1024,4096]},{"key":"option_question_projection.weight","shape":[1024,4096]},{"key":"global_projection.weight","shape":[1024,4096]},{"key":"option_context_projection.weight","shape":[1024,4096]},{"key":"option_lexical_projection.weight","shape":[1024,4096]},{"key":"type_embedding.weight","shape":[3,1024]},{"key":"evidence_layers.0.query_norm.weight","shape":[1024]},{"key":"evidence_layers.0.query_norm.bias","shape":[1024]},{"key":"evidence_layers.0.memory_norm.weight","shape":[1024]},{"key":"evidence_layers.0.memory_norm.bias","shape":[1024]},{"key":"evidence_layers.0.attention.in_proj_weight","shape":[3072,1024]},{"key":"evidence_layers.0.attention.in_proj_bias","shape":[3072]},{"key":"evidence_layers.0.attention.out_proj.weight","shape":[1024,1024]},{"key":"evidence_layers.0.attention.out_proj.bias","shape":[1024]},{"key":"evidence_layers.0.feedforward_norm.weight","shape":[1024]},{"key":"evidence_layers.0.feedforward_norm.bias","shape":[1024]},{"key":"evidence_layers.0.feedforward.0.weight","shape":[4096,1024]},{"key":"evidence_layers.0.feedforward.0.bias","shape":[4096]},{"key":"evidence_layers.0.feedforward.3.weight","shape":[1024,4096]},{"key":"evidence_layers.0.feedforward.3.bias","shape":[1024]},{"key":"evidence_layers.1.query_norm.weight","shape":[1024]},{"key":"evidence_layers.1.query_norm.bias","shape":[1024]},{"key":"evidence_layers.1.memory_norm.weight","shape":[1024]},{"key":"evidence_layers.1.memory_norm.bias","shape":[1024]},{"key":"evidence_layers.1.attention.in_proj_weight","shape":[3072,1024]},{"key":"evidence_layers.1.attention.in_proj_bias","shape":[3072]},{"key":"evidence_layers.1.attention.out_proj.weight","shape":[1024,1024]},{"key":"evidence_layers.1.attention.out_proj.bias","shape":[1024]},{"key":"evidence_layers.1.feedforward_norm.weight","shape":[1024]},{"key":"evidence_layers.1.feedforward_norm.bias","shape":[1024]},{"key":"evidence_layers.1.feedforward.0.weight","shape":[4096,1024]},{"key":"evidence_layers.1.feedforward.0.bias","shape":[4096]},{"key":"evidence_layers.1.feedforward.3.weight","shape":[1024,4096]},{"key":"evidence_layers.1.feedforward.3.bias","shape":[1024]},{"key":"option_summary_norm.weight","shape":[1024]},{"key":"option_summary_norm.bias","shape":[1024]},{"key":"layers.0.self_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.0.self_attn.in_proj_bias","shape":[3072]},{"key":"layers.0.self_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.0.self_attn.out_proj.bias","shape":[1024]},{"key":"layers.0.multihead_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.0.multihead_attn.in_proj_bias","shape":[3072]},{"key":"layers.0.multihead_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.0.multihead_attn.out_proj.bias","shape":[1024]},{"key":"layers.0.linear1.weight","shape":[4096,1024]},{"key":"layers.0.linear1.bias","shape":[4096]},{"key":"layers.0.linear2.weight","shape":[1024,4096]},{"key":"layers.0.linear2.bias","shape":[1024]},{"key":"layers.0.norm1.weight","shape":[1024]},{"key":"layers.0.norm1.bias","shape":[1024]},{"key":"layers.0.norm2.weight","shape":[1024]},{"key":"layers.0.norm2.bias","shape":[1024]},{"key":"layers.0.norm3.weight","shape":[1024]},{"key":"layers.0.norm3.bias","shape":[1024]},{"key":"layers.1.self_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.1.self_attn.in_proj_bias","shape":[3072]},{"key":"layers.1.self_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.1.self_attn.out_proj.bias","shape":[1024]},{"key":"layers.1.multihead_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.1.multihead_attn.in_proj_bias","shape":[3072]},{"key":"layers.1.multihead_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.1.multihead_attn.out_proj.bias","shape":[1024]},{"key":"layers.1.linear1.weight","shape":[4096,1024]},{"key":"layers.1.linear1.bias","shape":[4096]},{"key":"layers.1.linear2.weight","shape":[1024,4096]},{"key":"layers.1.linear2.bias","shape":[1024]},{"key":"layers.1.norm1.weight","shape":[1024]},{"key":"layers.1.norm1.bias","shape":[1024]},{"key":"layers.1.norm2.weight","shape":[1024]},{"key":"layers.1.norm2.bias","shape":[1024]},{"key":"layers.1.norm3.weight","shape":[1024]},{"key":"layers.1.norm3.bias","shape":[1024]},{"key":"layers.2.self_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.2.self_attn.in_proj_bias","shape":[3072]},{"key":"layers.2.self_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.2.self_attn.out_proj.bias","shape":[1024]},{"key":"layers.2.multihead_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.2.multihead_attn.in_proj_bias","shape":[3072]},{"key":"layers.2.multihead_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.2.multihead_attn.out_proj.bias","shape":[1024]},{"key":"layers.2.linear1.weight","shape":[4096,1024]},{"key":"layers.2.linear1.bias","shape":[4096]},{"key":"layers.2.linear2.weight","shape":[1024,4096]},{"key":"layers.2.linear2.bias","shape":[1024]},{"key":"layers.2.norm1.weight","shape":[1024]},{"key":"layers.2.norm1.bias","shape":[1024]},{"key":"layers.2.norm2.weight","shape":[1024]},{"key":"layers.2.norm2.bias","shape":[1024]},{"key":"layers.2.norm3.weight","shape":[1024]},{"key":"layers.2.norm3.bias","shape":[1024]},{"key":"layers.3.self_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.3.self_attn.in_proj_bias","shape":[3072]},{"key":"layers.3.self_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.3.self_attn.out_proj.bias","shape":[1024]},{"key":"layers.3.multihead_attn.in_proj_weight","shape":[3072,1024]},{"key":"layers.3.multihead_attn.in_proj_bias","shape":[3072]},{"key":"layers.3.multihead_attn.out_proj.weight","shape":[1024,1024]},{"key":"layers.3.multihead_attn.out_proj.bias","shape":[1024]},{"key":"layers.3.linear1.weight","shape":[4096,1024]},{"key":"layers.3.linear1.bias","shape":[4096]},{"key":"layers.3.linear2.weight","shape":[1024,4096]},{"key":"layers.3.linear2.bias","shape":[1024]},{"key":"layers.3.norm1.weight","shape":[1024]},{"key":"layers.3.norm1.bias","shape":[1024]},{"key":"layers.3.norm2.weight","shape":[1024]},{"key":"layers.3.norm2.bias","shape":[1024]},{"key":"layers.3.norm3.weight","shape":[1024]},{"key":"layers.3.norm3.bias","shape":[1024]},{"key":"field_norm.weight","shape":[1024]},{"key":"field_norm.bias","shape":[1024]},{"key":"option_norm.weight","shape":[1024]},{"key":"option_norm.bias","shape":[1024]},{"key":"residual_scorer.0.weight","shape":[1024,4096]},{"key":"residual_scorer.0.bias","shape":[1024]},{"key":"residual_scorer.3.weight","shape":[1,1024]},{"key":"residual_scorer.3.bias","shape":[1]}],"mha_in_proj_split":{"evidence_layers.0.attention.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"evidence_layers.1.attention.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.0.self_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.0.multihead_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.1.self_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.1.multihead_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.2.self_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.2.multihead_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.3.self_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"},"layers.3.multihead_attn.in_proj_weight":{"shape":[3072,1024],"split":"rows [0:1024] = W_q, [1024:2048] = W_k, [2048:3072] = W_v (q = x @ W_q.T + b_q)"}},"module_map":{"evidence_layers.N":"EvidenceRoutingLayer: query_norm, memory_norm (LayerNorm 1024), attention = nn.MultiheadAttention(1024, 16, batch_first) [in_proj_weight 3072x1024, in_proj_bias 3072, out_proj.weight 1024x1024, out_proj.bias 1024], feedforward_norm, feedforward.0 Linear 1024->4096, GELU (exact erf), feedforward.3 Linear 4096->1024; pre-LN, cross-attn only (no option-option attention)","layers.N":"nn.TransformerDecoderLayer(1024, 16, 4096, gelu, norm_first, batch_first): norm1 -> self_attn (fields), norm2 -> multihead_attn (memory), norm3 -> linear1 / gelu / linear2; each MHA has in_proj_weight 3072x1024 + in_proj_bias + out_proj","projections":"hidden_norm LayerNorm 4096; memory/question/option_question/global/option_context/option_lexical projections 4096->1024, no bias","scorer":"residual_scorer.0 Linear 4096->1024, GELU, residual_scorer.3 Linear 1024->1","scalars":"prior_logit_scale, joint_logit_scale (clamped at ln 100 then exp), residual_gate (sigmoid)"},"author_head_vs_oracle_logits_max_abs":9.5367431640625e-07,"core_eager_vs_author_head":{"max_abs_logit_diff":4.76837158203125e-07,"note":"own_t04: T 493 padded to 512 (key padding mask), O 11 + 1 padding option","max_abs_prob_diff":6.51925802230835e-09},"author_head_export":{"ok":false,"error_type":"GuardOnDataDependentSymNode","first_error_line":"torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not extract specialized integer from data-dependent expression u0 (unhinted: u0).  (Size-like symbols: none)","error_head":"torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not extract specialized integer from data-dependent expression u0 (unhinted: u0).  (Size-like symbols: none)\nCaused by: (hf/hub/models--Cloudflare--clef-flash/snapshots/17f0b0ad64efb65d273590632833508766b2aae6/joint_schema_model.py:355 in forward)\nFor more information, run with TORCH_LOGS=\"dynamic\"\nFor extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL=\"u0\"\nIf you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1\nFor more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing"},"core_export":{"ok":true,"seconds":0.33443498611450195,"aten_ops":{"<built-in function getitem>":24,"aten._assert_tensor_metadata.default":1,"aten.abs.default":1,"aten.add.Tensor":24,"aten.bitwise_not.default":2,"aten.cat.default":1,"aten.clamp.default":2,"aten.clamp_min.default":2,"aten.contiguous.default":10,"aten.cosine_similarity.default":1,"aten.div.Tensor":3,"aten.dropout.default":23,"aten.embedding.default":1,"aten.exp.default":2,"aten.expand.default":10,"aten.expand_as.default":2,"aten.gelu.default":7,"aten.index_select.default":1,"aten.layer_norm.default":22,"aten.le.Scalar":1,"aten.linalg_vector_norm.default":2,"aten.linear.default":46,"aten.masked_fill.Scalar":2,"aten.masked_fill_.Scalar":10,"aten.matmul.default":7,"aten.mul.Tensor":5,"aten.nan_to_num.default":1,"aten.permute.default":10,"aten.reshape.default":11,"aten.scaled_dot_product_attention.default":10,"aten.select.int":27,"aten.sigmoid.default":1,"aten.softmax.int":1,"aten.split_with_sizes.default":12,"aten.squeeze.dim":11,"aten.sub.Tensor":2,"aten.sum.default":1,"aten.sum.dim_IntList":1,"aten.to.dtype":1,"aten.transpose.int":70,"aten.unflatten.int":10,"aten.unsqueeze.default":20,"aten.view.default":100,"aten.zeros_like.default":10},"has_native_multi_head_attention":false,"has_scaled_dot_product_attention":true,"has_data_dependent":false,"exported_vs_eager_max_abs":4.76837158203125e-07,"core_aten_ops_after_decompositions":{"<built-in function getitem>":46,"aten._assert_tensor_metadata.default":1,"aten._softmax.default":11,"aten._to_copy.default":1,"aten.abs.default":1,"aten.add.Tensor":34,"aten.addmm.default":40,"aten.any.dim":10,"aten.bitwise_not.default":2,"aten.bmm.default":20,"aten.cat.default":1,"aten.clamp.default":6,"aten.clone.default":45,"aten.div.Tensor":5,"aten.embedding.default":1,"aten.eq.Scalar":12,"aten.exp.default":2,"aten.expand.default":52,"aten.full_like.default":20,"aten.gelu.default":7,"aten.index_select.default":1,"aten.isnan.default":1,"aten.le.Scalar":1,"aten.linalg_vector_norm.default":4,"aten.logical_not.default":20,"aten.mm.default":13,"aten.mul.Scalar":20,"aten.mul.Tensor":6,"aten.native_layer_norm.default":22,"aten.permute.default":156,"aten.scalar_tensor.default":15,"aten.select.int":27,"aten.sigmoid.default":1,"aten.split_with_sizes.default":12,"aten.squeeze.dims":12,"aten.sub.Tensor":2,"aten.sum.dim_IntList":3,"aten.unsqueeze.default":21,"aten.view.default":237,"aten.where.self":25}},"seconds":2.457181930541992},"joint_head_header":{"file":"joint_head.safetensors","file_bytes":243538016,"header_json_bytes":12368,"metadata":{"format":"pt"},"config":{"hidden_size":4096,"width":1024,"routing_layers":2,"layers":4,"heads":16,"feedforward":4096},"tensor_count":122,"total_params":121762820,"total_bytes":243525640,"dtype_counts":{"BF16":122},"by_module":{"evidence_layers.0":{"tensors":14,"params":12598272},"evidence_layers.1":{"tensors":14,"params":12598272},"field_norm":{"tensors":2,"params":2048},"global_projection":{"tensors":1,"params":4194304},"hidden_norm":{"tensors":2,"params":8192},"joint_logit_scale":{"tensors":1,"params":1},"layers.0":{"tensors":18,"params":16796672},"layers.1":{"tensors":18,"params":16796672},"layers.2":{"tensors":18,"params":16796672},"layers.3":{"tensors":18,"params":16796672},"memory_projection":{"tensors":1,"params":4194304},"option_context_projection":{"tensors":1,"params":4194304},"option_lexical_projection":{"tensors":1,"params":4194304},"option_norm":{"tensors":2,"params":2048},"option_question_projection":{"tensors":1,"params":4194304},"option_summary_norm":{"tensors":2,"params":2048},"prior_logit_scale":{"tensors":1,"params":1},"question_projection":{"tensors":1,"params":4194304},"residual_gate":{"tensors":1,"params":1},"residual_scorer":{"tensors":4,"params":4196353},"type_embedding":{"tensors":1,"params":3072}}}},"key_block_probes":{"what":"the first bucket export was exact at t512 / t1024 / t2048 and wrong (and different on every call) at t4096; the probes below isolate it to the plain softmax(q k^T * s [+ bias]) v chain at 4,032 keys or more on this GPU stack, each op alone exact, and keys in blocks of <= 2,048 with one shared max exact (the variant 'split2'); clef_head.py uses that form above 2,048 keys (KEY_CHUNK)","t4096_probe":[{"check":"eager","run":"own_t01/text","T_pad":2048,"max_abs_dlogit_vs_oracle":1.1920928955078125e-06},{"check":"eager","run":"own_t01/text","T_pad":4096,"max_abs_dlogit_vs_oracle":1.1920928955078125e-06},{"check":"eager","run":"own_t15/text","T_pad":4096,"max_abs_dlogit_vs_oracle":7.152557373046875e-07},{"check":"aot","run":"own_t01/text","function":"t2048","max_abs_dlogit_vs_eager":[4.76837158203125e-07,4.76837158203125e-07,4.76837158203125e-07],"calls_bit_equal":[true,true],"max_abs_dlogit_vs_oracle":[9.5367431640625e-07,9.5367431640625e-07,9.5367431640625e-07]},{"check":"aot","run":"own_t01/text","function":"t4096","max_abs_dlogit_vs_eager":[0.9460321664810181,0.7688549757003784,0.3479567766189575],"calls_bit_equal":[false,false],"max_abs_dlogit_vs_oracle":[0.9460312128067017,0.768854022026062,0.3479558229446411]},{"check":"aot","run":"own_t15/text","function":"t4096","max_abs_dlogit_vs_eager":[0.5649429559707642,0.5881619453430176,0.5035417079925537],"calls_bit_equal":[false,false],"max_abs_dlogit_vs_oracle":[0.5649431943893433,0.5881621837615967,0.5035419464111328]}],"tmax_probe":[{"check":"aot","run":"own_t01/text","function":"t3072","max_abs_dlogit_vs_eager":[4.76837158203125e-07,4.76837158203125e-07,4.76837158203125e-07],"calls_bit_equal":[true,true],"max_abs_dlogit_vs_oracle":[9.5367431640625e-07,9.5367431640625e-07,9.5367431640625e-07]},{"check":"aot","run":"own_t01/text","function":"t4032","max_abs_dlogit_vs_eager":[1.6366243362426758,1.6021692752838135,1.6248557567596436],"calls_bit_equal":[false,false],"max_abs_dlogit_vs_oracle":[1.6366255283355713,1.602170467376709,1.624856948852539]},{"check":"aot","run":"own_t01/text","function":"t4096","max_abs_dlogit_vs_eager":[0.6377487182617188,0.977328896522522,1.2278081178665161],"calls_bit_equal":[false,false],"max_abs_dlogit_vs_oracle":[0.6377496719360352,0.9773279428482056,1.2278071641921997]},{"check":"aot","run":"own_t15/text","function":"t3072","max_abs_dlogit_vs_eager":[1.6093254089355469e-06,1.6093254089355469e-06,1.6093254089355469e-06],"calls_bit_equal":[true,true],"max_abs_dlogit_vs_oracle":[9.5367431640625e-07,9.5367431640625e-07,9.5367431640625e-07]},{"check":"aot","run":"own_t15/text","function":"t4032","max_abs_dlogit_vs_eager":[0.0905836820602417,0.08952242136001587,0.08823966979980469],"calls_bit_equal":[false,false],"max_abs_dlogit_vs_oracle":[0.0905836820602417,0.08952218294143677,0.08823919296264648]},{"check":"aot","run":"own_t15/text","function":"t4096","max_abs_dlogit_vs_eager":[0.46283331513404846,0.5406434535980225,0.5014500617980957],"calls_bit_equal":[false,false],"max_abs_dlogit_vs_oracle":[0.4628335237503052,0.5406436920166016,0.5014503002166748]}],"head_debug_intermediates":[{"T":3072,"call":0,"rel_max_abs_diff":{"memory":"3.63e-07","qv":"2.58e-07","ctx":"1.05e-07","global_vector":"1.62e-09","queries_l0":"2.70e-07","routed":"2.10e-07","summaries":"4.67e-07","fields":"4.46e-07","logits":"9.51e-08"}},{"T":3072,"call":1,"rel_max_abs_diff":{"memory":"3.63e-07","qv":"2.58e-07","ctx":"1.05e-07","global_vector":"1.62e-09","queries_l0":"2.70e-07","routed":"2.10e-07","summaries":"4.67e-07","fields":"4.46e-07","logits":"9.51e-08"}},{"T":3072,"call":2,"rel_max_abs_diff":{"memory":"3.63e-07","qv":"2.58e-07","ctx":"1.05e-07","global_vector":"1.62e-09","queries_l0":"2.70e-07","routed":"2.10e-07","summaries":"4.67e-07","fields":"4.46e-07","logits":"9.51e-08"}},{"T":4096,"call":0,"rel_max_abs_diff":{"memory":"3.63e-07","qv":"2.58e-07","ctx":"1.05e-07","global_vector":"1.62e-09","queries_l0":"1.36e-01","routed":"1.50e-01","summaries":"8.19e-02","fields":"2.52e-01","logits":"1.18e-01"}},{"T":4096,"call":1,"rel_max_abs_diff":{"memory":"3.63e-07","qv":"2.58e-07","ctx":"1.05e-07","global_vector":"1.62e-09","queries_l0":"1.05e-01","routed":"1.32e-01","summaries":"8.51e-02","fields":"2.44e-01","logits":"1.30e-01"}},{"T":4096,"call":2,"rel_max_abs_diff":{"memory":"3.63e-07","qv":"2.58e-07","ctx":"1.05e-07","global_vector":"1.62e-09","queries_l0":"1.22e-01","routed":"1.32e-01","summaries":"9.01e-02","fields":"2.77e-01","logits":"1.38e-01"}}],"attn_probe":[{"H":16,"O":128,"T":3072,"scores_elements":6291456,"max_abs_diff":[4.76837158203125e-07,4.76837158203125e-07,4.76837158203125e-07],"calls_bit_equal":[true,true]},{"H":16,"O":128,"T":3584,"scores_elements":7340032,"max_abs_diff":[3.2782554626464844e-07,3.2782554626464844e-07,3.2782554626464844e-07],"calls_bit_equal":[true,true]},{"H":16,"O":128,"T":4032,"scores_elements":8257536,"max_abs_diff":[0.4729096591472626,0.4729096591472626,0.4729096591472626],"calls_bit_equal":[false,false]},{"H":16,"O":128,"T":4096,"scores_elements":8388608,"max_abs_diff":[0.4592271149158478,0.4592271149158478,0.4592271149158478],"calls_bit_equal":[false,false]},{"H":16,"O":64,"T":4096,"scores_elements":4194304,"max_abs_diff":[0.6726735830307007,0.5111886262893677,0.4910769462585449],"calls_bit_equal":[false,false]},{"H":16,"O":16,"T":4096,"scores_elements":1048576,"max_abs_diff":[0.6251742243766785,0.6251742243766785,0.6251742243766785],"calls_bit_equal":[false,false]},{"H":8,"O":128,"T":4096,"scores_elements":4194304,"max_abs_diff":[0.3677171468734741,0.4740341603755951,0.4740341603755951],"calls_bit_equal":[false,false]},{"H":16,"O":128,"T":8192,"scores_elements":16777216,"max_abs_diff":[0.3889436423778534,0.6586601138114929,0.4595509171485901],"calls_bit_equal":[false,false]},{"H":1,"O":128,"T":4096,"scores_elements":524288,"max_abs_diff":[0.41875237226486206,0.26554954051971436,0.41875237226486206],"calls_bit_equal":[false,false]}],"attn_fix_probe":[{"variant":"fused","T":3584,"max_abs_diff":["3.73e-07","3.73e-07","3.73e-07"],"calls_bit_equal":[true,true]},{"variant":"fused","T":4032,"max_abs_diff":["4.89e-01","4.89e-01","4.89e-01"],"calls_bit_equal":[false,false]},{"variant":"fused","T":4096,"max_abs_diff":["4.85e-01","4.48e-01","4.43e-01"],"calls_bit_equal":[false,false]},{"variant":"nobias","T":3584,"max_abs_diff":["3.28e-07","3.28e-07","3.28e-07"],"calls_bit_equal":[true,true]},{"variant":"nobias","T":4032,"max_abs_diff":["1.27e-01","1.27e-01","1.27e-01"],"calls_bit_equal":[false,false]},{"variant":"nobias","T":4096,"max_abs_diff":["2.18e-01","2.18e-01","2.31e-01"],"calls_bit_equal":[false,false]},{"variant":"split2","T":3584,"max_abs_diff":["4.17e-07","4.17e-07","4.17e-07"],"calls_bit_equal":[true,true]},{"variant":"split2","T":4032,"max_abs_diff":["5.66e-07","5.66e-07","5.66e-07"],"calls_bit_equal":[true,true]},{"variant":"split2","T":4096,"max_abs_diff":["5.66e-07","5.66e-07","5.66e-07"],"calls_bit_equal":[true,true]}],"op_probe":[{"op":"softmax","T":3584,"rel_max_abs_diff":["3.29e-06","3.29e-06","3.29e-06"],"calls_bit_equal":[true,true]},{"op":"softmax","T":4032,"rel_max_abs_diff":["2.88e-06","2.88e-06","2.88e-06"],"calls_bit_equal":[true,true]},{"op":"softmax","T":4096,"rel_max_abs_diff":["2.76e-06","2.76e-06","2.76e-06"],"calls_bit_equal":[true,true]},{"op":"av","T":3584,"rel_max_abs_diff":["2.82e-06","2.82e-06","2.82e-06"],"calls_bit_equal":[true,true]},{"op":"av","T":4032,"rel_max_abs_diff":["2.50e-06","2.50e-06","2.50e-06"],"calls_bit_equal":[true,true]},{"op":"av","T":4096,"rel_max_abs_diff":["2.77e-06","2.77e-06","2.77e-06"],"calls_bit_equal":[true,true]},{"op":"qk","T":3584,"rel_max_abs_diff":["0.00e+00","0.00e+00","0.00e+00"],"calls_bit_equal":[true,true]},{"op":"qk","T":4032,"rel_max_abs_diff":["0.00e+00","0.00e+00","0.00e+00"],"calls_bit_equal":[true,true]},{"op":"qk","T":4096,"rel_max_abs_diff":["0.00e+00","0.00e+00","0.00e+00"],"calls_bit_equal":[true,true]},{"op":"maxsum","T":3584,"rel_max_abs_diff":["9.24e-08","9.24e-08","9.24e-08"],"calls_bit_equal":[true,true]},{"op":"maxsum","T":4032,"rel_max_abs_diff":["1.49e-07","1.49e-07","1.49e-07"],"calls_bit_equal":[true,true]},{"op":"maxsum","T":4096,"rel_max_abs_diff":["1.37e-07","1.37e-07","1.37e-07"],"calls_bit_equal":[true,true]}]},"diag_long":{"decoder_readout":{"schema":"clef-flash-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle ids, oracle fp32 image_embeds -> fp16, zero-padded to 1024 rows), the fp16 hidden read through the author's fp32 head","bundle":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","name":"clef_flash_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"metadata_sha256":"bbd4401e72dd055c9fb9998aecf3fcd1b43ea789698bec9040c0d89412a77ae3","main_mlirb":{"bytes":15880253283,"sha256":"0b7dfa30a0597ead745d8f3ed9ed11fa5ac639a4c578a33a7cca7ced11b9bea8"},"tokenizer_sha256":{"chat_template.jinja":"a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715","tokenizer.json":"06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523","tokenizer_config.json":"91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"},"aimodelc":{"path":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","bytes":29731951203,"tree_sha256":"196c089dca5446037f583f9957a5fa5eac5717324e7b49dfce7cc9b7fd49ec9e","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bbf348d5a22a3d8b3e86844153f727c37e891c6f4e2d9f6fc5408413c2947104","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"6ce9393c561b16b7b0fe36956f3ba3746597ba9d7b27fc9542ded93258bec117","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"f602c49d232fd7d08d9b02a276cfcbc033fe27920f1f50d5fa645ad7a1093273","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e60f6614c8f00859c2a8dd86e0b8e8275956de95b7530c1c94692111cba0d556","main-h16c.mlirb":"0d85c1b244198a3102e1b37d1ad76b9ecba06424607c232697c7ab66c60a08ad","main.hash":"5a75a90e1c5d061241566adf4a940ede53aea86252b35ac78d0afd8889ba401b","metadata.json":"2c6acb63c7baf6f6072fb2d06b15d6163e95d83c56c05e302c79c90060ffe786","stats.json":"c4eac63e4b03f3d77179ecfe949163a5b64281cca93484f54a2a9ba9c813e50c"}}},"chunk":64,"max_ctx":4096,"readout":"fresh zero states; a row of T ids runs as ceil(T / 64) calls of 'main' (static S = 64): call k gets ids[64k : 64k + 64] with position_ids 0..64k+63; the last call is padded with <|endoftext|> (248044) and the hidden rows of the padded positions are discarded (causal: they cannot reach a real position). The hidden rows [1, 64, 4096] of every call, concatenated and cut to T, are the backbone's final-norm last_hidden_state [T, 4096] the head reads.","descriptor":{"function":"main","inputs":{"input_ids":[[1,64],"int32"],"position_ids":[[1,-1],"int32"],"image_embeds":[[1024,4096],"float16"],"image_rc":[[1024,2],"int32"],"rope_shift_start":[[1],"int32"],"rope_shift_amount":[[1],"int32"]},"outputs":{"hidden":[[1,64,4096],"float16"]},"states":{"keyCache":[[8,1,4,-1,256],"float16"],"valueCache":[[8,1,4,-1,256],"float16"],"convState":[[24,1,8192,3],"float16"],"recState":[[24,1,32,128,128],"float16"]}},"descriptor_same_in_every_process":true,"contract":{"inputs":{"input_ids":[[1,64],"int32"],"position_ids":[[1,-1],"int32"],"image_embeds":[[1024,4096],"float16"],"image_rc":[[1024,2],"int32"],"rope_shift_start":[[1],"int32"],"rope_shift_amount":[[1],"int32"]},"outputs":{"hidden":[[1,64,4096],"float16"]},"states":{"keyCache":[[8,1,4,-1,256],"float16"],"valueCache":[[8,1,4,-1,256],"float16"],"convState":[[24,1,8192,3],"float16"],"recState":[[24,1,32,128,128],"float16"]}},"oracle":{"path":"<lane>/oracle/diag_long/records_oracle.json","sha256":"a82663c78090b34e4b65ac91206fe81907ac3b4cc7fe7a43921f14ec4fc73ba0","source":"Cloudflare/clef-flash@17f0b0ad64efb65d273590632833508766b2aae6","versions":{"python":"3.12.11","torch":"2.9.0","torchvision":"0.24.0","transformers":"5.17.0","pillow":"12.3.0","numpy":"2.5.3","huggingface_hub":"1.33.0"}},"subset":"all","runs_file":null,"script":{"path":"conversion/clef_flash/readout_gate.py","sha256":"ae633622829f02e83a202d522994559c77ab25a8e08dc759ec14435c983c50f0"},"skipped":[],"bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs of the mean |dp| over all (question, option) entries of the run"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the author's JointSchemaHead, fp32, CPU (parity_decoder_torch.AuthorHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"other_gpu_processes":{"note":"process listing removed before publication (5 lines); it showed other sessions' GPU work at the time = the times are contended"},"processes":[{"shard":"shard_00","runs":["diag_l4k/text/base"],"prompts":2,"pid":23812,"load_seconds":2.869299541984219,"load_split_seconds":{"model":0.005567083979258314,"main":2.863732458004961},"function_names":["main"],"process_wall_seconds":26.3991640000022,"reset_check":{"run":["diag_l4k","text","base"],"bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":8.706443792005302},"contract_mismatch":[]}],"summary":{"runs":1,"expected_runs":1,"missing_runs":[],"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0006085634231567383,"max_abs_dp_non_near_tie":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.183874499285594e-05,"mean_question_max_abs_dp":0.00014693811535835267,"max_abs_dlogit":0.005308389663696289,"min_pos_cos":0.9585155997026775,"positions":4050,"positions_below":{"0.99":1,"0.9":0,"0.5":0},"positions_below_0.99_image_rows":0,"runs_with_positions_below_0.99":["diag_l4k/text"],"hidden_max_abs_diff":3.510885238647461,"hidden_max_rel_diff":0.06571105875942301,"worst_run":{"id":"diag_l4k","arm":"text","max_abs_dp":0.0006085634231567383},"finite_all":true,"tokens":4050,"calls":64,"ms_per_call_median":135.68272950942628,"ms_per_call_warm_median":null},"by_source_arm":{"diag_long/text":{"runs":1,"expected_runs":1,"missing_runs":[],"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0006085634231567383,"max_abs_dp_non_near_tie":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.183874499285594e-05,"mean_question_max_abs_dp":0.00014693811535835267,"max_abs_dlogit":0.005308389663696289,"min_pos_cos":0.9585155997026775,"positions":4050,"positions_below":{"0.99":1,"0.9":0,"0.5":0},"positions_below_0.99_image_rows":0,"runs_with_positions_below_0.99":["diag_l4k/text"],"hidden_max_abs_diff":3.510885238647461,"hidden_max_rel_diff":0.06571105875942301,"worst_run":{"id":"diag_l4k","arm":"text","max_abs_dp":0.0006085634231567383},"finite_all":true,"tokens":4050,"calls":64,"ms_per_call_median":135.68272950942628,"ms_per_call_warm_median":null}},"near_ties":[],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal_all_processes":true},"result":"PASS","red_arm":null,"timing":{"contended":true,"S":64,"load_seconds":{"first_process":2.869299541984219,"median":2.869299541984219,"all":[2.869299541984219]},"ms_per_call_warm_runs":{"n":0,"median":null,"p10":null,"p90":null},"ms_per_call_all_runs":{"n":64,"median":135.68272950942628,"p10":134.69260869605932,"p90":137.28060879802797},"first_call_of_process_ms":[2853.520374977961],"ms_per_token_warm_runs":{"n":0,"median":null,"p10":null,"p90":null},"run_decode_seconds_warm_runs":{"n":0,"median":null,"p10":null,"p90":null},"run_wall_seconds_warm_runs":{"n":0,"median":null,"p10":null,"p90":null},"by_source_arm_warm":{}},"seconds":{"gpu_processes":26.43442729098024,"scoring":18.768715457990766,"total":45.25934954098193},"runs":[{"id":"diag_l4k","arm":"text","variant":"base","tokens":4050,"calls":64,"padded_tokens":4096,"grid":null,"n_image_rows":0,"start":1073741824,"amount":0,"image_tokens":0,"decode_seconds":11.969020374992397,"wall_seconds":11.981537542014848,"finite":true,"index":0,"shard":"shard_00","first_in_process":true,"source":"diag_long","hidden":{"max_abs_diff":3.510885238647461,"ref_absmax":53.42913818359375,"rel_max_abs_diff":0.06571105875942301,"min_pos_cos":0.9585155997026775,"min_pos_cos_index":1300,"mean_pos_cos":0.9999498341249335,"positions":4050,"positions_below":{"0.99":1,"0.9":0,"0.5":0},"positions_below_0.99_image_rows":0},"questions":[{"question_id":"renewal_term","type":"choice","n_options":4,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.4901161193847656e-05,"mean_abs_dp":7.454509614035487e-06,"max_abs_dlogit":0.0026395320892333984,"oracle_top2_margin":0.9872398688457906,"near_tie":false,"probs":[0.9915643930435181,0.004301857203245163,0.0023620231077075005,0.0017717141890898347]},{"question_id":"late_interest","type":"choice","n_options":4,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.1219253540039062e-05,"mean_abs_dp":1.059012720361352e-05,"max_abs_dlogit":0.003871917724609375,"oracle_top2_margin":0.9902353805955499,"near_tie":false,"probs":[0.0032059953082352877,0.9934060573577881,0.002177524147555232,0.0012103483313694596]},{"question_id":"data_owner","type":"noul","n_options":2,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.759539246559143e-05,"mean_abs_dp":6.75935298204422e-05,"max_abs_dlogit":0.004309535026550293,"oracle_top2_margin":0.9558184500783682,"near_tie":false,"probs":[0.9778416156768799,0.02215835265815258]},{"question_id":"chair_elected","type":"choice","n_options":3,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.2411346435546875e-05,"mean_abs_dp":1.4965888112783432e-05,"max_abs_dlogit":0.005107879638671875,"oracle_top2_margin":0.983835079241544,"near_tie":false,"probs":[0.0036849775351583958,0.0062513346783816814,0.9900637269020081]},{"question_id":"risk","type":"score","n_options":3,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006085634231567383,"mean_abs_dp":0.0004057188828786214,"max_abs_dlogit":0.005308389663696289,"oracle_top2_margin":0.15503713488578796,"near_tie":false,"probs":[0.1562410295009613,0.499744713306427,0.3440142571926117]}],"argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006085634231567383,"mean_abs_dp":9.183874499285594e-05,"max_abs_dlogit":0.005308389663696289}],"note":"round 5 diagnostic: one 4,050-token row (64 calls of S=64; the last two attend 4,032 / 4,096 keys)","generated_at":"2026-10-03T05:22:45+00:00"},"head_h2":{"schema":"clef-flash-head-h2/1","gate":"end to end in Python: the decoder bundle's fp16 hidden (readout shards, not re-run) -> host span matrices + fp16-table lexical -> head graph -> per-question fp32 softmax","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02},"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite_and_padding_zero":true,"rerun_bit_equal":true},"result":"PASS","summary":{"runs":1,"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.182832582155243e-05,"max_abs_dlogit":0.005308628082275391,"worst_run":"diag_l4k/text"},"by_source_arm":{"diag_long/text":{"runs":1,"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.182832582155243e-05,"max_abs_dlogit":0.005308628082275391,"worst_run":"diag_l4k/text"}},"head_bundle":{"dir":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","metadata_sha256":"631219aafcdb56ef1f008e5476f0e7a282933abdc51b4835132676d97124ab20","aimodelc":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_metadata":{"graph":"conversion/clef_flash/clef_head.py ClefHeadGraph: the author's JointSchemaHead for one record, tensor inputs only","shape":"bucket","functions":["t512","t1024","t2048","t4096"],"buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"dims":null,"host_pad":"bucket","host_pad_rule":"T to the smallest bucket >= T (512, 1024, 2048, 4096), Q to 16, O to 128; padding rows / columns zero, their valid flags 0","inputs":{"hidden":"[T, 4096] f32","key_valid":"[T] f32","q_avg":"[Q, T] f32","o_avg":"[O, T] f32","g_avg":"[1, T] f32","lexical":"[O, 4096] f32","member":"[Q, O] f32","type_ids":"[Q] i32","q_valid":"[Q] f32","o_valid":"[O] f32"},"input_order":["hidden","key_valid","q_avg","o_avg","g_avg","lexical","member","type_ids","q_valid","o_valid"],"outputs":{"logits":"[O] f32; padding options 0"},"weights":"fp16w32","softmax":"host: per question, fp32, over that question's options","lexical":"host: lm_head_fp16.bin rows of each option span's ids, averaged (clef_head.lexical_rows)","source":{"hf_id":"Cloudflare/clef-flash","revision":"17f0b0ad64efb65d273590632833508766b2aae6","joint_schema_model_sha256":"0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3","joint_head_safetensors_sha256":"19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0","joint_head_config_sha256":"77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0"}}},"descriptors":{"t512":{"function":"t512","inputs":{"hidden":[[512,4096],"float32"],"key_valid":[[512],"float32"],"q_avg":[[16,512],"float32"],"o_avg":[[128,512],"float32"],"g_avg":[[1,512],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t1024":{"function":"t1024","inputs":{"hidden":[[1024,4096],"float32"],"key_valid":[[1024],"float32"],"q_avg":[[16,1024],"float32"],"o_avg":[[128,1024],"float32"],"g_avg":[[1,1024],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t2048":{"function":"t2048","inputs":{"hidden":[[2048,4096],"float32"],"key_valid":[[2048],"float32"],"q_avg":[[16,2048],"float32"],"o_avg":[[128,2048],"float32"],"g_avg":[[1,2048],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}},"t4096":{"function":"t4096","inputs":{"hidden":[[4096,4096],"float32"],"key_valid":[[4096],"float32"],"q_avg":[[16,4096],"float32"],"o_avg":[[128,4096],"float32"],"g_avg":[[1,4096],"float32"],"lexical":[[128,4096],"float32"],"member":[[16,128],"float32"],"type_ids":[[16],"int32"],"q_valid":[[16],"float32"],"o_valid":[[128],"float32"]},"outputs":{"logits":[[128],"float32"]}}},"descriptors_same_every_process":true,"oracle":{"path":"<lane>/oracle/diag_long/records_oracle.json","sha256":"a82663c78090b34e4b65ac91206fe81907ac3b4cc7fe7a43921f14ec4fc73ba0","runs":1},"table":{"path":"<lane>/exports/host/lm_head_fp16.bin","json":"<lane>/exports/host/lm_head_fp16.json"},"near_ties":[],"timing":{"contended":true,"by_T":{"T>1500":{"runs":1,"median_tokens":4050.0,"median_padded_T":4096.0,"first_call_ms_median":42.954708012985066,"second_call_ms_median":19.260166998719797,"host_ms_median":14.385291986400262}},"all_first_call_ms_median":42.954708012985066,"all_second_call_ms_median":19.260166998719797},"processes":[{"pid":26013,"started":1791004988.50686,"load_model_seconds":0.006020583998179063,"function_names":["t512","t1024","t2048","t4096"],"load_function_seconds":{"t512":0.32548333299928345,"t1024":0.001373584003886208,"t2048":0.0016910830163396895,"t4096":0.0010930410062428564},"reset_check":{"run":["diag_l4k","text","base"],"bit_equal":true,"max_abs_diff":0.0},"finished":1791004988.97965,"process_wall_seconds":0.9375499580055475,"runs":1}],"red_arm":null,"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max"},"script":{"gate_head.py":"1d66d14945d23d15b80866799a6287a08fcca41cf95ff56ea41e418a8c0b5e0b","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d"},"seconds":{"gpu_processes":0.9389422079839278},"generated_at":"2026-10-03T05:23:09+00:00","runs":[{"id":"diag_l4k","arm":"text","variant":"base","tokens":4050,"function":"t4096","shape":{"T":4096,"Q":16,"O":128},"options":16,"layout":[[0,4],[4,8],[8,10],[10,13],[13,16]],"host_ms":14.385291986400262,"calls_bit_equal":true,"finite":true,"padding_logits_zero":true,"index":0,"source":"diag_long","argmax_all_equal":true,"argmax_equal_non_near_tie":true,"max_abs_dp":0.0006085634231567383,"mean_abs_dp":9.182832582155243e-05,"max_abs_dlogit":0.005308628082275391,"author_head_fp32_same_hidden":{"max_abs_dp":0.0006085634231567383,"mean_abs_dp":9.183874499285594e-05},"vs_author_head_same_hidden_max_abs_dp":1.1920928955078125e-07,"argmax_equal_author_head_same_hidden":true}],"readout":{"path":"<lane>/results/readout_diag_long_fp16_pf64.json","sha256":"bc651ee470dbec4a2e1c19485766decdd0529194549723d747877d158a690b7e","bundle":"clef_flash_decode_fp16_pf64","result":"PASS","author_head_summary":{"max_abs_dp":0.0006085634231567383,"mean_of_run_mean_abs_dp":9.183874499285594e-05,"argmax_equal_non_near_tie":5,"questions_non_near_tie":5,"argmax_equal_near_tie":0,"near_tie_questions":0},"vs_author_head_same_hidden":{"max_abs_dp":1.1920928955078125e-07,"argmax_equal_runs":1}}},"decide_py":{"schema":"clef-flash-decide-e2e/1","what":"decide.py from the raw request (host ids, host image preprocess + tower AOT, decoder AOT, head AOT, fp16 lm_head table) vs the author's fp32 systemone() response of the same record","assets":{"bundle":"<lane>/exports/bundles/clef_flash_decode_fp16_pf64","decoder_aot":"<lane>/exports/bundles_aotc/clef_flash_decode_fp16_pf64.h16c.aimodelc","head_bundle":"<lane>/exports/head/clef_flash_head_bucket_fp16w32","head_aot":"<lane>/exports/head/clef_flash_head_bucket_fp16w32/clef_flash_head_bucket_fp16w32.h16c.aimodelc","head_buckets":[["t512",512],["t1024",1024],["t2048",2048],["t4096",4096]],"table":"<lane>/exports/host/lm_head_fp16.bin","table_sha256_recorded":"bfccf00b8d5d3f6d810839a52c2ec627700a9aaebaa26aedcb96931bd21ba36d","towers":{"256":"<lane>/exports/clef_flash_g256_vision_fp16w32_aotc/clef_flash_g256_vision_fp16w32.h16c.aimodelc","448":"<lane>/exports/clef_flash_g448_vision_fp16w32_aotc/clef_flash_g448_vision_fp16w32.h16c.aimodelc"},"decoder_bundle_name":"clef_flash_decode_fp16_pf64","decoder_prefill_chunk":64},"load_seconds":{"decoder_s":2.747313166008098,"head_s":0.0736841670004651,"tower_g256_s":1.280893292016117,"tower_g448_s":1.2031085409980733},"oracle":"<lane>/oracle/diag_long/records_oracle.json","fixtures":"<lane>/fixtures/diag_long/records.json","summary":{"requests":1,"answer_values_equal":3,"answers":5,"fields_equal":12,"fields":20,"choices_equal":"3/3","max_abs_diff":0.0005999999999999894,"ids_equal_all":true,"spans_equal_all":true,"model_and_usage_equal_all":true},"script_sha256":{"decide.py":"dd05d8481286810d19329e9f0d2b9cf93b391fec1884ffb038ef68622a207d71","clef_head.py":"f548f87d6fed83ae82a5a66c87626f7e155401755aa383e1adb2eea25f88687d","host.py":"356ae136ee62252a052a23bd13415228281065980754603aa89d75e225eb8e2a"},"timing_note":"contended GPU (other sessions); first request of the process includes warm-up","generated_at":"2026-10-03T05:23:24+00:00","runs":[{"id":"diag_l4k","arm":"text","source":"diag_long","tokens":4050,"wall_seconds":9.498165458993753,"ids_equal_oracle":true,"spans_equal_oracle":true,"hidden_min_pos_cos_vs_oracle":0.9585155997026775,"response":{"model":"clef-flash","answers":{"renewal_term":{"type":"choice","choice":"12","confidence":0.9916,"probabilities":{"6":0.0018,"12":0.9916,"24":0.0043,"36":0.0024}},"late_interest":{"type":"choice","choice":"1.5","confidence":0.9934,"probabilities":{"1":0.0032,"1.5":0.9934,"2":0.0022,"5":0.0012}},"data_owner":{"type":"noul","noul":0.9778},"chair_elected":{"type":"choice","choice":"somerfield","confidence":0.9901,"probabilities":{"ollerton":0.0037,"somerfield":0.9901,"reyes":0.0063}},"risk":{"type":"score","score":1.1878,"confidence":0.4997,"legend":{"0":"unfavourable","1":"balanced","2":"favourable"},"probabilities":{"0":0.1562,"1":0.4997,"2":0.344}}},"usage":{"input_tokens":4050,"output_tokens":0}},"oracle_response":{"model":"clef-flash","answers":{"renewal_term":{"type":"choice","choice":"12","confidence":0.9915,"probabilities":{"6":0.0018,"12":0.9915,"24":0.0043,"36":0.0024}},"late_interest":{"type":"choice","choice":"1.5","confidence":0.9934,"probabilities":{"1":0.0032,"1.5":0.9934,"2":0.0022,"5":0.0012}},"data_owner":{"type":"noul","noul":0.9779},"chair_elected":{"type":"choice","choice":"somerfield","confidence":0.9901,"probabilities":{"ollerton":0.0037,"somerfield":0.9901,"reyes":0.0063}},"risk":{"type":"score","score":1.1873,"confidence":0.4991,"legend":{"0":"unfavourable","1":"balanced","2":"favourable"},"probabilities":{"0":0.1568,"1":0.4991,"2":0.3441}}},"usage":{"input_tokens":4050,"output_tokens":0}},"compare":{"fields":20,"fields_equal":12,"answer_values":[{"question_id":"renewal_term","type":"choice","oracle":"12","mine":"12","equal":true},{"question_id":"late_interest","type":"choice","oracle":"1.5","mine":"1.5","equal":true},{"question_id":"data_owner","type":"noul","oracle":0.9779,"mine":0.9778,"equal":false},{"question_id":"chair_elected","type":"choice","oracle":"somerfield","mine":"somerfield","equal":true},{"question_id":"risk","type":"score","oracle":1.1873,"mine":1.1878,"equal":false}],"answer_values_equal":3,"answers":5,"choices_equal":"3/3","model_equal":true,"usage_equal":true,"max_abs_diff":0.0005999999999999894,"diffs":[{"question_id":"renewal_term","field":"confidence","oracle":0.9915,"mine":0.9916,"abs_diff":9.999999999998899e-05},{"question_id":"renewal_term","field":"p[12]","oracle":0.9915,"mine":0.9916,"abs_diff":9.999999999998899e-05},{"question_id":"data_owner","field":"noul","oracle":0.9779,"mine":0.9778,"abs_diff":9.999999999998899e-05},{"question_id":"risk","field":"score","oracle":1.1873,"mine":1.1878,"abs_diff":0.0004999999999999449},{"question_id":"risk","field":"confidence","oracle":0.4991,"mine":0.4997,"abs_diff":0.0005999999999999894},{"question_id":"risk","field":"p[0]","oracle":0.1568,"mine":0.1562,"abs_diff":0.0005999999999999894},{"question_id":"risk","field":"p[1]","oracle":0.4991,"mine":0.4997,"abs_diff":0.0005999999999999894},{"question_id":"risk","field":"p[2]","oracle":0.3441,"mine":0.344,"abs_diff":0.0001000000000000445}]},"trace":{"host_ids_ms":4.638166981749237,"tokens":4050,"decoder":{"calls":64,"S":64,"call_ms":[521.7802090046462,133.98570800200105,134.56216698978096,134.56920799217187,134.40679101040587,135.0003330153413,135.21891701384448,135.33995798206888,134.74970898823813,134.24708301317878,134.3576670042239,134.70208301441744,134.4929999904707,135.07945800665766,135.2510419965256,133.83508400875144,134.29308298509568,134.98312499723397,135.50704199587926,135.60124998912215,134.86258301418275,135.2627919986844,135.76750000356697,135.75325001147576,135.37000000360422,135.51820800057612,135.69908399949782,135.57695798226632,135.4643750237301,135.39258297532797,136.26895801280625,135.7098330045119,136.16104098036885,135.41333400644362,136.34029199602082,136.85412498307414,135.64762499299832,136.93083301768638,136.1124999821186,135.49395799054764,136.37333302176557,136.32595899980515,136.3154579885304,135.85320799029432,136.42445800360292,136.70416598324664,136.97187500656582,136.05699999607168,136.51366598787718,136.8463750113733,136.41725000343286,136.59725000616163,136.78508301381953,136.5578330005519,136.93895799224265,136.9598750025034,136.55116700101644,136.81866700062528,136.6098749858793,137.54495800822042,137.02049999847077,136.73979198210873,137.07875000545755,137.14891698327847],"total_ms":9077.717121952446},"head":{"function":"t4096","host_ms":6.328666990157217,"head_ms":32.88320801220834},"total_ms":9139.393459015992}}]}}}