Core AI model zoo

Qwen3.6-27B (text decoder) — Core AI

The Mac-class dense companion to Qwen3.6-35B-A3B. Source: Qwen/Qwen3.6-27B (an image-text-to-text checkpoint; this card is the text decoder).

Where the 35B-A3B is a sparse MoE, the 27B is the same Qwen3.5 hybrid decoder run dense — no experts, no router, just the proven token mixers at scale. 64 layers on a 3:1 interleave of GatedDeltaNet linear-attention mixers and gated full attention:

27B parameters, all dense → the entire model is read per token. Unlike the 35B-A3B (≈3B active), there is no sparsity to hide behind, so this is a true 27B-class decode: the quality of a large dense model, at the memory-bandwidth speed that implies on a Mac.

Why it runs on this Mac today: head_dim 256 keeps the full-attention Q buffer small, so it side-steps the macOS-27-beta MPSGraph decode-heap bug that currently blocks the head_dim-512 Gemma 4 12B. The qwen3.5 family already runs on this engine.

⬇️ Converted .aimodel bundle: qwen3_6_27b_decode_int8hu_block32_sym/ (28 GB, full LanguageBundle incl. tokenizer; decode-only loop-free for the pipelined engine). HF upload user-gated.

Use it

⚡ One line — run the kit’s task op on this model (import CoreAIOps; no session, no model plumbing, downloads on first use):

let tldr = try await CoreAI.summarize(text, options: .model("qwen3.6-27b"))

Twenty ops, one shape — Cookbook.

▶️ Run it (source) — the ChatDemo runner (GUI + CLI, one app for every chat model in the catalog):

git clone https://github.com/john-rocky/coreai-kit
open coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj
# → Run, then pick "Qwen3.6-27B (dense)" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/ChatDemo
swift run chat-cli --model qwen3.6-27b --prompt "What can you do, offline?"

💻 Build with it — complete; the glue is kit API, copy-paste runs:

import CoreAIKit

let chat = try await ChatSession(catalog: "qwen3.6-27b")
let reply = try await chat.respond(to: prompt)
// reply: the answer, generated fully on-device

Also runs behind Apple’s FoundationModels API — CoreAIKit’s KitLanguageModel plugs this bundle into the system LanguageModelSession; capabilities (tool calling, guided generation) auto-detect per model.

The take-home is Examples/ChatDemo/Sources/QuickStart.swift — this exact code as one typed function, no UI; the CLI is an argument shell over it, and the GUI drives the same ChatSession across turns for its transcript. Multi-turn? Hold the ChatSession and call respond(to:) per turn — it keeps the conversation history; streamResponse(to:) yields tokens as they decode.

Integration checklist

Measured (macOS 27 beta, M4 Max 128 GB, release llm-benchmark, COREAI_CHUNK_THRESHOLD=1)

config bundle prefill tok/s decode tok/s numerics (teacher-forced vs bf16 HF oracle)
int8 linear per-block-32 + untied absmax int8 head (int8hu --head-sym) = SHIP 28 GB 15.8 15.9 int8 == full precision at every confident position (see below)

Numerics — 27B fp32 would need ~111 GB RAM (27.8B × 4 B), unsafe alongside the OS on a 128 GB host, so the oracle is the checkpoint’s native bf16; the gate is teacher-forced single-step argmax under the oracle-margin≥0.1 rule (the rule absorbs bf16↔fp16 rounding on sub-0.1-margin top-2 ties). The result is cleaner than the 35B-A3B’s:

int4 is a borderline speed/size option — not the quality ship. A linear int4 per-block-32 gate (head kept fp16) also scores 15/16 under the rule, but unlike int8 it pays a real fidelity cost: it flips a high-confidence position (pos 0, margin 0.688) that fp16 and int8 both get right, at cos 0.985, and its per-position cosine is systematically lower across the board (~0.985–0.999 vs int8’s 0.998–0.99998). It is far better than the 35B-A3B’s int4 (9–10/16 NO-GO — the shared hybrid body tolerates 4-bit better when dense), so int4 is usable for a ~14 GB bundle at roughly double the decode rate, but it is a measurable quality drop. We ship int8hu as the default; int4 is one export flag away (int4lin) for callers who want size/speed over the last bit of fidelity.

A mixed-precision middle ground was tested and rejected: MLP/FFN linears (~63 % of the per-token read) at int4-asymmetric with attention / GatedDeltaNet / head / edge-layers kept at int8. Keeping the mixers at int8 does repair int4lin’s high-confidence flip — confirming the attention/GDN path, not the FFN bulk, is the 4-bit-sensitive part — but the int4 MLP then introduces its own confident flip at a different (middle-layer) position that keeping the edge layers at int8 does not fix. The result lands in the same borderline-quality class as plain int4lin while being slower (~23 vs ~30 tok/s) and larger (~19 vs ~14 GB), so there is no quality-preserving speedup between int8 (clean, 15.9 tok/s) and int4 (borderline, ~30): int8 is the quality ship and int4 the size/speed option, with nothing useful in between.

How to reproduce

cd coreai-models   # with the qwen3_5 model overlay (see ../conversion)
# convert (CPU-side; ~54 GB fp16 load + in-RAM int8 quantize, fits a 128 GB host)
.venv/bin/python ../coreai-models-community/conversion/export_qwen3_5_decode_pipelined.py \
    int8hu --head-sym --hf-id Qwen/Qwen3.6-27B
# bench (needs the coreai-pipelined extra-states engine patch + COREAI_CHUNK_THRESHOLD=1)
COREAI_CHUNK_THRESHOLD=1 .build/release/llm-benchmark \
    --model exports/qwen3_6_27b_decode_int8hu_block32_sym -p 64 -g 128 -n 3

Model overlay: models/macos/qwen3_5.py (the shared Qwen3.5 hybrid decoder — the GVA query/key head-repeat is config-driven, so the 27B’s 48v/16k ratio needs no new code; the loader picks up the untied lm_head.weight from the checkpoint root). Decode-only loop-free because the GatedDeltaNet while_loop doesn’t lower on the GPU delegate. No MoE files — that is the 35B-A3B’s qwen3_5_moe.py; this dense port reuses the base directly.

This site is open source. Improve this page.