.aimodel → CoreAIKit → device → publishedThe repeatable flow for taking a model from “converted” to “shipped on iPhone, measured, and published,” validated end-to-end on Parakeet-TDT-0.6B in one session (convert → gate → Swift engine → app → device 47.9× real-time → HF → zoo card → post). The per-topic docs go deep on each stage; this is the runbook + the cross-cutting traps that don’t live in any single one.
Deep docs per stage: ML convert → conversion-guide.md; target/precision →
compute-units-and-authoring.md; Swift runtime →
swift-runtime.md; LanguageModelSession → fm-provider.md;
on-device AOT → aot-and-specialization.md.
echo <tag> > ~/code/coreai/_GPU_LOCK before any python/AOT GPU run,
rm after; run GPU work solo (the beta driver kernel-panics under parallel GPU load).transformers than the
export env); the main coreai-models/.venv for export + gating. Don’t cross-contaminate.git add EXPLICIT paths only (repos carry unrelated WIP). Never commit models/weights/
.aimodel/.aimodelc/build files/large .npz. No “claude” in messages or committer.1. Convert + gate the ML. Re-author in plain torch from model.safetensors, export with
export_to_coreai, gate token-exact (or per-token cosine) vs a saved HF golden (oracle.npz).
Don’t move on until the .aimodel matches the golden on GPU. (conversion-guide.md)
2. Pin host pre/post-processing in NumPy BEFORE writing Swift. Anything the host computes (mel,
image norm, detok, samplers) — reimplement the exact algorithm the Swift will run (e.g. a manual
cos/sin-DFT, not torch.stft) in NumPy, gate it token-exact end-to-end, and diff vs the golden
features. This is where Parakeet’s hidden normalization bug surfaced — see traps. Scripts:
gate_mel_swift.py / mel_swift_sim.py / diff_swift_mel.py in conversion/parakeet/.
3. CoreAIKit engine. New Kit<X>Model in coreai-kit/Sources/CoreAIKit/<X>/: load each graph
with GraphModel(contentsOf:computeUnits:.gpu), tokenizer via AutoTokenizer.from(modelFolder:),
host loop in Swift. Mirror the closest sibling’s public surface (KitWhisperModel / KitASRModel /
VoxCPM2TTS). Bundle fixed matrices (mel filterbanks) as a target resource. swift build
--target CoreAIKit to typecheck (catches await/convenience init/API misuse cheaply).
4. App wiring. Add the engine to the view model (enum case + load + run branch). For models with a
big graph, add an iOS sideload override: if Documents/Models/<X>/ holds the bundle, load it
(init(bundleAt:)); else Hub-download. The picker/UI usually needs no change.
5. Headless self-test bench (the perf number). An env-gated entrypoint (<X>_SELFTEST=1, launched
from App.init() via Task.detached) that: resolves the bundle (sideloaded on device, else local
artifacts on Mac), loads a clip, times load + N transcribe/generate runs (run 1 cold, rest warm),
computes RTF (audio_sec ÷ run_sec), and writes Documents/<x>_selftest_result.txt + NSLog. Run it
on Mac first (must be token-exact) before the device.
6. On-device ship. Build the app for the device (xcodebuild -destination 'platform=iOS,id=<UDID>'
-configuration Release), devicectl device install app, … process launch. If a 1 GB+ graph’s
on-device JIT stalls, AOT-compile it and sideload (see traps). Run the self-test on device:
xcrun devicectl device copy to --device <UDID> --domain-type appDataContainer \
--domain-identifier <bid> --source <file> --destination "Documents/Models/<X>/<name>"
xcrun devicectl device process launch --device <UDID> -e '{"<X>_SELFTEST":"1"}' <bid>
xcrun devicectl device copy from --device <UDID> --domain-type appDataContainer \
--domain-identifier <bid> --source "Documents/<x>_selftest_result.txt" --destination /tmp/r.txt
7. Publish (USER-GATED). HF upload (conversion/_<x>_hf_upload.py: stage → upload_folder;
patch tokenizer_class at stage time). zoo/<x>.md (pipeline, graph contracts, on-device speed,
“lessons”, convert-yourself) + a models/README.md row + a root README.md row (don’t forget this
one). Commit (explicit paths). Draft the X post with the measured RTF — post is the user’s.
ParakeetFeatureExtractor always
per-utterance normalizes (the do_normalize arg is dead code) and the bucket must be filled by
silence-padding the audio (zero-padding the mel makes the decoder hallucinate). Caught only
because step 2 gated the NumPy mel e2e first.xcrun coreai-build compile <m>.aimodel --output <dir> --platform iOS
--architecture h18p --preferred-compute gpu --min-deployment-version 27.0 → <m>.h18p.aimodelc
(embeds a precompiled MPSGraph; ~2× the .aimodel size). Arch tracks the device-identifier major
version (iPhone 17 Pro = iPhone18,1 → h18p; M-series Mac → h16c), NOT the marketing name.
Small graphs (≤~50 MB) JIT fine — ship them as portable .aimodel. (aot-and-specialization.md)tokenizer_class (unsupportedTokenizer). Retag the
bundle’s tokenizer_config.json to a registered class (PreTrainedTokenizer → BPETokenizer);
decode is driven by tokenizer.json’s decoder, so it stays exact. Do it in the upload script.mlx-swift whose mlx submodule clone looks “stuck” — 0% CPU on git index-pack). Reuse the
Debug build’s -derivedDataPath so packages are already resolved.devicectl facts: install preserves the app data container (sideloaded files survive a
reinstall); env vars need the -e '{"K":"V"}' JSON flag; copy individual files (bulk can
false-succeed). GraphModel/AIModel(contentsOf:) load .aimodel or .aimodelc.Bundle.module / cross-file symbols show as SourceKit errors in-editor until a real build
regenerates the resource accessor — swift build is the source of truth, not the squiggles.