Core AI model zoo

coreai-cli — the engineering record

README.md is the user’s document and the PyPI page. This is the rest: why the commands are shaped the way they are, the logs that back each claim, what each one caught, and how a release is cut. Nothing here is needed to use the tool.

Where this lives, and why

cli/ inside the zoo, alongside the conversion/ scripts and the knowledge/ notes the rules are transcribed from. That is deliberate and reversible:

Path resolution is location-independent — find_zoo_root() walks up for the directory holding both models/ and conversion/, and falls back to the sibling layout — so lifting cli/ out into a standalone repo later is a directory move and nothing else. Do that when export routes beyond the set Apple and the zoo already cover; until then a standalone repo would be a thinner front door than this one.

The wheel is stdlib-only for local targets and ships these files plus the dated zoo-routes snapshot (coreai_zoo_routes.py, regenerated by make_zoo_routes.py), nothing else.

Releasing

python3 cli/release.py 0.2.0 --ios-build 24A353 --dry-run   # every step, no changes
python3 cli/release.py 0.2.0 --ios-build 24A353             # the release

One command: regenerate the routes snapshot, run selftest.py, read the host (sw_vers, Xcode, coreai-build, the runtime wheels, the phone’s build from --ios-build), retitle CHANGELOG.md’s [Unreleased] section and stamp its Validated on line, update the same line in README.md, bump pyproject.toml, build the wheel and sdist, twine check. It refuses to stamp an Apple seed build (26A5xxx) as validated unless told --allow-beta, because a release that says “validated on macOS 27” has to have been validated on the release OS. Commit, tag and twine upload are printed, not run — the upload is a public act. Keep CHANGELOG.md’s [Unreleased] section current as changes land; the release only retitles it.

export — how the numbers were established

The zoo number was 13 until the routing pass. Not because 42 models were unroutable — because recipe.toml records hf_repo, which is the output repo, and only 13 recipes happened to name the model they convert from. The other 42 record it somewhere else: the export script’s argparse default, or the model card’s upstream link. Reading those three places instead of one closes it. The durable fix is a source_hf_id field in the zoo’s recipe schema; this reads around its absence rather than editing that repo.

The iOS cliff is a real raise site: raise ValueError("Model 'gemma3' does not support iOS variant") at export/pipeline.py:150, reached only after AutoConfig has read the checkpoint. export is convert-only — no AIModel.load, no SpecializationOptions anywhere in export/pipeline.py — so it does not contend for the exclusive GPU. Running the bundle does, which is why verify checks the lock and export does not.

log what it shows
logs/export-routing-cases.txt seven targets covering every branch: preset (exit 0), generic-with-iOS (0), zoo ×2 (0), iOS-cliff (2), silent-preflight (1), no-route (2)
logs/export-commands-resolve.txt the emitted commands run through the exporter’s own --dry-run and resolve; plus the pipeline.py:150 raise site behind the BLOCKED claim
logs/end-to-end-qwen3-0.6b.txt export --run producing a real bundle, which doctor found clean and verify gated 16/16

verify — what it caught on day one

The canonical gate prompt — "The capital of France is", the one the zoo recommended and shipped as the default — fails its own margin rule at n=16 on Qwen3-0.6B: positions 1 and 5 sit at 0.0885 and 0.0041. It is deterministic at the first token and not over a 16-token continuation, because after “Paris.” the model free-runs into a list where the next country is a near-tie. The recommendation and the margin rule were both in the notes and had conflicted at n=16 for as long as both existed. Nothing surfaced it until a tool checked the prompt instead of trusting it.

Fixed 2026-08-01 in both coreai_verify.py and the zoo’s conversion/coreai_gate.py, by measurement (the table in the README). Note the old default is not broken everywhere — gemma-3 clears it comfortably — which is what made it survive: whether it gates or silently refuses depends on the model under test. Reciting a fixed sequence holds because there is nothing to free-run into once the answer is given; counting drifts once the numbers get long enough to admit a second plausible formatting.

coreai_gate.py covers 7 zoo families and cannot gate a stock-recipe bundle at all; the stock backend is that gap.

log what it shows
logs/end-to-end-qwen3-0.6b.txt export → doctor → verify on one model: real bundle, lint clean, 16/16 token-exact against the fp32 oracle
logs/verify-validation.txt the rejected canonical prompt (exit 3), the full passing run (exit 0), and the zoo delegation
logs/verify-qwen3-0.6b-transcript.json the transcript: input ids, both sides’ output, per-step margins, verdict, environment
logs/selftest.txt the verdict rule, both sides of the margin floor

doctor — the sweep, the false positives, the catches

A sweep over all 90 local bundles reported findings on 41. Every finding class in that sweep was hand-verified against the artifact before the README claimed anything; the false positives found on the way (a vision encoder held to the LanguageBundle contract, a ship manifest that merely shares the name metadata.json, a Jinja template that renders eos_token mid-expression, and the working div(x, 2, rounding_mode="floor") form) are fixed and covered by fixtures. A lint that flags the documented workaround is worse than no lint.

First real catch: three published repos. The sweep found EOS-NOT-EMITTED-BY-TEMPLATE live on Hugging Face — gemma-3-4b-it-CoreAI-official, gemma-3-12b-it-CoreAI-official and functiongemma-270m-coreml all declared eos_token: "<eos>" (id 1, document end) while their chat template ends a turn with <end_of_turn> (id 106, and upstream generation_config.eos_token_id is [1, 106]). Any runtime that derives its stop token from eos_token alone — swift-transformers does — generated to the token cap. Fixed 2026-07-31 by logs/fix_gemma_eos.py (one field, byte-range replacement, verified on re-read); logs/audit_gemma_eos.py re-swept all 18 published Gemma tokenizer configs afterwards: none fire.

Worth noting what the audit also corrected. GEMMA4_12B_STATE.md had warned since July that the published Gemma-4 12B/31B bundles still carried the old <eos>; they did not — the note was stale, and acting on it would have been wasted work. Reading the artifacts beat reading the note about the artifacts.

The same sweep flagged CHAT-TEMPLATE-MISSING on the four legacy gemma-4-E{2,4}B*-coreml ports, which shipped a tokenizer and no template — a runtime applying one had nothing to apply and fell back to raw completion without a word. Fixed 2026-07-31 by logs/fix_gemma4_coreml_chat.py, which ships google/gemma-4-{E2B,E4B}-it’s own template verbatim. The two repos that still report no chat template are the embeddinggemma repos — an embedding model has no chat surface, which is the rule reporting correctly, not a gap.

The host-build rules (0.2.0), and the premise that did not survive measurement. IR-040-DEBUG-LOC said a 0.4.0-era asset “loads on beta 1 and nowhere after”, and the 0.2.0 kickoff asked for it to be relaxed because Apple’s OS 27 beta 5 release notes list the incident (177008303) as fixed. Before relaxing it, the zoo’s new load sweep (conversion/zoo_smoke.py) downloaded a published 0.4.0-era bundle (qwen3.5-0.8B-CoreAI/gpu-pipelined/qwen3_5_0_8b_decode_int8hu_perchan_sym) and loaded it on macOS 27 26A5416b with coreai-core 1.0.0b2: AIModel.load aborted with the July signature (expected AICode versioned location → LLVM ERROR: cannot unwrap empty odiec_module_t), and xcrun coreai-build compile (3600.82.1) failed identically while compiling the bundle’s b2 twin fine. The release note was not a measurement. So the rule is now host-conditional in the other direction: info on beta 1, fatal on every build from beta 2 on, and a release build inherits nothing — IR040_MEASURED_OK_FROM in coreai_doctor.py names the first build a sweep measures to load such an asset, and stays None until one does. The same producer audit (--all --stamp-only) found 34 published bundles still carrying 0.4.0-era IR that the July recovery left in place because the catalog did not serve them; they are the strip_debug_info queue.

AOTC-STALE-TOOLCHAIN was wrong in the other direction: it called any AOT artifact older than the installed toolchain fatal, which is every published h18p artifact (compiled on beta 3, loading on every later device build). The incident it encodes (181264112) is specifically a beta-2-or-earlier coreai-build, so that is now the fatal case; merely older than the installed toolchain is info, and whether a release toolchain still specializes a beta-3 artifact is a device check, not a lint.

log what it shows
logs/case-a-known-broken.txt a 0.4.0-era bundle: 1 fatal, 1 runaway, exit 2
logs/case-a-ground-truth-load-abort.txt the same asset actually aborting at AIModel.load, so the fatal is not an assertion
logs/case-b-known-good.txt the device-gated nanbeige4.2-3B ship bundle: 0 defects, 4 ship requirements, exit 0
logs/case-c-source-lint-rf-detr.txt the source lint over stock transformers RF-DETR, independently re-finding the patterns that port hit
logs/case-d-checkpoint-wna8o8.txt the Gemma-4 mobile QAT checkpoint, flagged from its safetensors headers before any export
logs/case-e-published-gemma3-eos.txt the eos rule swept across all 18 published Gemma tokenizer configs, after the fix — none fire
logs/selftest.txt 18 fixture checks over 16 source rules

eval — validation

End to end on qwen3-0.6b (4-bit, macOS bundle, llm-runner), GSM8K, --thinking off:

check result
same settings twice every row identical, delta B - A = +0.0%, protocol matched
--max-new-tokens 512 vs 256 REFUSED, naming max_new_tokens
halving the budget truncated items 1 → 2, as it should

The accuracy itself was 0/10, and the interesting part is why the tool says so: 1 item ran out of budget and 7 finished without the marker. Qwen3-0.6B answers in \boxed{0}, not #### 0, in a third of its budget. Raising the budget would not move that number by one item, and an earlier version of the README said “raise the budget” anyway — it counted every missing marker as truncation. A real run is what exposed it; the split between truncated and off-format exists because of that run.

The template digest in the protocol is taken from the rendered prefix, measured on Qwen3-0.6B:

--thinking rendered assistant prefix digest
on …<\|im_start\|>assistant\n 7e77fde99496
off …assistant\n<think>\n\n</think>\n\n 5c8507f2b86b
default same as on 7e77fde99496

The zoo’s existing device batch format (g4out.json) carries ids, not text, and eval refuses it rather than growing a tokenizer — the moment scoring owns one, the two arms are no longer scored by identical code.

Self-test

python3 cli/selftest.py — the decision rules, negative half first: 18 fixture checks over 16 source rules (each rule must fire on its trigger and stay silent on the documented workaround), 8 over verify’s verdict (both sides of the margin floor), 20 over eval’s refusals, 29 over the host-build comparator and the severity of the two beta-5 rules from beta 1 through a release build. release.py runs it and will not release on a failure.

This site is open source. Improve this page.