CoreAIKit

Examples

Start here: OpsGallery — all twenty task ops as cards, rendered from CoreAI.Op.allCases: pick an input, run the one-line call, see the result. The fastest way to find out what the kit can do on your own data.

Building any GUI example: xcodegen generate (brew install xcodegen), open the project, and either pick your team in Signing & Capabilities or export DEVELOPMENT_TEAM=XXXXXXXXXX before generating — no project hard-codes one.

Every other example app in one place, on two shelves:

Model runners

App Kind What it runs
Transcribe speech-to-text Any asr catalog model (Whisper / Qwen3-ASR / Parakeet): record, choose, or demo clip → transcript. GUI + swift run transcribe-cli.
Tidy ASR text normalization Any textNormalizer catalog model (S1-mini by Superwhisper): raw dictation → written text, steered by the model’s styling / structure / context axes. GUI + swift run tidy-cli.
ChatDemo chat Minimal multiplatform chat over ChatSession; pick a starter model, it downloads and streams.
VLChat vision-language Any vlm catalog model (Qwen3-VL 2B/4B/8B, Holo2 GUI grounding): pick a photo, ask about it. GUI + swift run vlchat-cli.
Look vision-language Two screens over MiniCPM-V 4.7 1B on the device. Ask: one photo + one typed question → one answer, with the first-token time and tok/s the app measured. Watch: rules in words (“a chessboard” → a chime) checked on the camera, a picked video or the example photos one frame at a time; a rule whose answer turns yes fires its reaction (a sound, a haptic, a spoken line, a timer, a count, a log line). Example photos (fetched by tools/fetch_examples.sh). GUI (iPhone) + an iPhone UI test.
Cutout segmentation + matting WeVisOne-VL-2B on the device: a photo + the words for what to cut out → each match’s box and alpha matte, the cut-out saved as a PNG with transparency (to Photos on an iPhone, to a file on a Mac); about 1 s for one match on an iPhone 18 Pro. Runs the model on the system CoreAI framework itself (the decoder’s hidden state feeds the pixel decoder); the model folder comes from the Hugging Face repository mlboydaisuke/WeVisOne-VL-2B-CoreAI and is copied in by hand (no catalog id). GUI (iPhone + Mac) + swift run cutout-cli + an iPhone UI test.
Speak text-to-speech Any tts catalog model (VoxCPM / VoxCPM2 / Kokoro): type a sentence, hear it. GUI + swift run speak-cli.
Music text-to-music Any music catalog model (Stable Audio): prompt → 44.1 kHz stereo. GUI + swift run music-cli.
AudioChat audio understanding Any audio catalog model (Qwen2.5-Omni): load a clip, ask “what do you hear?”. GUI + swift run audiochat-cli (macOS).
DepthCamera depth Any depth catalog model: live camera depth via CameraFeed. GUI + swift run depth-cli.
ActionCamera video Any video catalog model (V-JEPA 2): a rolling 16-frame clip → live action label. GUI + swift run action-cli.
ReadDoc document OCR Any ocr catalog model (Unlimited-OCR): document image → structured markdown. GUI + swift run readdoc-cli.
DiffuseChat diffusion LM Any dllm catalog model (LLaDA-8B): watch the reply denoise in place (parallel fill-in, not left-to-right). GUI + swift run diffuse-cli.
DetectCamera detection Any detection catalog model (RF-DETR / YOLOX) behind one KitDetector. GUI + swift run detect-cli.
LiveCamera live camera + video The four live tasks as four tabs: watch() detection, watchDepth(), a trigger that gates a VLM, and scan(videoAt:) over a video file. Measured stats and the thermal governor are on screen. GUI (iPhone) + swift run live-cli.
UpscaleDemo super-resolution AdcSR ×4 super-resolution: pick a photo, tap Upscale. GUI + swift run upscale-cli.
PhotoSearch image-text embeddings Semantic photo search — the photo library indexed with CLIP.
Decide typed decisions Any chat catalog model as a decision model (MiniCPM5 2B default): a speech gate (yes/no per utterance), a clipboard check with Shortcuts actions, and a passage reranker — probabilities and milliseconds per decision, nothing generated. GUI + swift run decide-cli.
Week typed decisions Your own week (Calendar read-only, pasted lines or JSON) or a sample week: every event asked one question on decider-0.8b (what it needs before it), the events that need something gathered in one list, reminders when you ask. GUI (iPhone + Mac) + swift run week-cli.
DocSearch visual doc retrieval Any retrieval catalog model (ColModernVBERT): query + page images → MaxSim ranking, no OCR. GUI (iPhone) + swift run docsearch-cli.
TextClassify zero-shot text classification GLiNER2.5-Decide: labels you name at call time, several questions about one text in a single forward. Your own messages (pasted, or a .txt / .md / .csv file) or a sample inbox, sorted into categories you name, each with an urgency and a sentiment; By category puts the most urgent first. GUI (iPhone + Mac) + swift run textclassify-cli.

SDK-feature demos

App OS surface What it shows
SiriAsk Siri / App Intents “Hey Siri, Ask Gemma” → an on-device Gemma 4 answers, warm-resident.
SpotlightApp Spotlight indexing Ask your own files with your own local model.
SpotlightChat SpotlightSearchTool Local RAG through Apple’s Spotlight tool (WWDC26), driven by your model.
FMToolDemo FoundationModels Local tool calling behind Apple’s LanguageModelSession.
GuidedDemo FoundationModels Guided generation: logits masked through a JSON-schema grammar.
DocChat — On-device RAG end to end: notes indexed with EmbeddingGemma, answered by a local chat model.
AskVLM Visual Intelligence Qwen3-VL as its own Visual Intelligence tab.
VisualIntel Visual Intelligence Your own models behind system Visual Intelligence.

All examples build against the kit via path: ../.. — clone the repo and they can’t version-skew. GUI apps commit their generated .xcodeproj (source of truth = project.yml): clone → open → Run, no xcodegen install needed. CLI shells run headless on macOS with swift run — the agent-verifiable door.

This site is open source. Improve this page.