---
family: granite-speech
license: apache-2.0
model_id: granite-speech-5.0-2b
source_url: https://huggingface.co/litert-community/granite-speech-5.0-2b
task: automatic-speech-recognition
---

# granite-speech-5.0-2b

| | |
|---|---|
| **Task** | automatic-speech-recognition |
| **Family** | granite-speech |
| **Source** | https://huggingface.co/litert-community/granite-speech-5.0-2b |
| **License** | apache-2.0 |

## Conversion

| | |
|---|---|
| **Tool** | litert-torch 0.10.0 (export_hf for the Granite language model, the token embedding exported separately; the audio encoder as a PyTorch module traced with litert-torch) packed with litert-lm-builder 0.19.1; repro = hf-to-litertlm REPRODUCE.md, the granite-speech-5.0-2b.litertlm entry (card L204-L207) litert-torch 0.10.0, litert-lm-builder 0.19.1 (litert-torch 0.10.0's own packing calls a builder API that 0.19 no longer has); LiteRT-LM 0.19.1 for every measurement except the encoder-alone times (ai-edge-litert 2.3.0) (card L31, L203, L206) |
| **Command** | `ibm-granite/granite-speech-5.0-2b rev 98160169 -> the language model: export_hf, prefill signatures 64, 256 and 1,024, recipe dynamic_wi8_afp32, KV cache 4,096; the audio encoder: a PyTorch module computing what the source's feature extractor, encoder and projector compute on the unpadded clip, one 36-second window, float16 weights; the source's tokenizer.json byte for byte, a chat template rendering `USER: <|audio|>{instruction}\n ASSISTANT:`, stop token <|end_of_text|> (100257), model type generic_model (card L59, L202, L204-L205)` |
| **Quantization** | language model: int8 weights (one scale per output channel), KV cache of 4,096 tokens, the file asks the GPU for float32 activations; token embedding table int8; audio encoder: float16 weights computed in float32, one 36-second window, CPU only (card L49, L54-L56) |

## Artifacts

| File | SHA-256 | Size (MB) |
|---|---|---|
| `granite-speech-5.0-2b.litertlm` | `c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265` | 2714.69 |

## Performance

No benchmark data yet.

## Device runs (NPU / LiteRT-LM)

Model-level on-device results — measured behavior of this exact artifact on the recorded device, only meaningful together with that environment. Throughput is conditional on the measured prompt length (Prompt (tokens); '-' = the source stated none): rows differing only there are different measurements, not re-runs. Full records (failure classes, log evidence): `data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__galaxy-s26.json`, `data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json`, `data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__galaxy-s26.json`, `data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__pixel-11-pro.json`.

| Device | Accelerator | Status | Full delegation | Output match | Prompt (tokens) | Latency p50 (ms) | Prefill tok/s | Decode tok/s | TTFT (ms) | Peak mem (MB) | Environment | Date | Provenance |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 5160.15 | 6089.0358 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the CPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 4364.8 | 5120.2253 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-06 | measured |
| galaxy-s26 | cpu | fallback | no | pass | sig:ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU | - | - | - | - | 6163.4642 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-06 | measured |
| galaxy-s26 | cpu | fallback | no | - | 1024 | - | 137.9159 | 10.0332 | 7524.55 | 3280.6298 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | gpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the GPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 3827.3 | 4042.9609 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | gpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the GPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 7467.6 | 3866.0424 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-06 | measured |
| galaxy-s26 | gpu | fallback | no | pass | sig:ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU | - | - | - | - | 3852.2061 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-06 | measured |
| galaxy-s26 | gpu | fallback | no | - | 1024 | - | 226.5526 | 14.8803 | 4604.2 | 1133.8547 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| mac-studio-m4-max | cpu | pass | - | - | sig:ASR clip timing on the Mac: a 5.9 s LibriSpeech clip, a 29.3 s and a 10.0 s FLEURS en_us clip in that order, one streaming call per clip with a new conversation each, the model loaded once per process, the transcription instruction, greedy, at most 256 new tokens; the median of 3 processes; audio encoder on the CPU; ttft_ms = the 10.0 s clip's (the third conversation's) first streamed text, audio encoding included | - | - | - | 1405.3 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-06 | measured |
| mac-studio-m4-max | cpu | pass | - | pass | sig:ASR gate: the 220 clips (20 LibriSpeech dev-clean, 50 FLEURS en_us, 50 FLEURS ja_jp and 25 FLEURS clips each of French, German, Spanish and Portuguese), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; scored against the source model in transformers 5.22.1 on the CPU (English with Whisper's English normalizer, the other European languages with its basic normalizer, Japanese by characters); line = the source model + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU | - | - | - | - | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-06 | measured |
| mac-studio-m4-max | cpu | pass | - | - | 1024 | - | 500.55 | 26.65 | 2178.1 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-06 | measured |
| mac-studio-m4-max | gpu | pass | - | - | sig:ASR clip timing on the Mac: a 5.9 s LibriSpeech clip, a 29.3 s and a 10.0 s FLEURS en_us clip in that order, one streaming call per clip with a new conversation each, the model loaded once per process, the transcription instruction, greedy, at most 256 new tokens; the median of 3 processes; audio encoder on the CPU; ttft_ms = the 10.0 s clip's (the third conversation's) first streamed text, audio encoding included | - | - | - | 1180.4 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-06 | measured |
| mac-studio-m4-max | gpu | pass | - | pass | sig:ASR gate: the 220 clips (20 LibriSpeech dev-clean, 50 FLEURS en_us, 50 FLEURS ja_jp and 25 FLEURS clips each of French, German, Spanish and Portuguese), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; scored against the source model in transformers 5.22.1 on the CPU (English with Whisper's English normalizer, the other European languages with its basic normalizer, Japanese by characters); line = the source model + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU | - | - | - | - | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-06 | measured |
| mac-studio-m4-max | gpu | load_failed | no | - | sig:ASR with the audio encoder on the GPU too (audio_backend=GPU; the language model on the GPU): conversation creation for the gate's first clip | - | - | - | - | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-06 | measured |
| mac-studio-m4-max | gpu | pass | - | - | 1024 | - | 2351.68 | 84.18 | 447.3 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-06 | measured |
| pixel-11-pro | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the CPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 26545.4 | 5330.817 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-07 | measured |
| pixel-11-pro | cpu | fallback | no | pass | sig:ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU | - | - | - | - | 6064.3779 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-06 | measured |
| pixel-11-pro | cpu | fallback | no | - | 1024 | - | 29.7512 | 4.7182 | 34984.75 | 2956.1733 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-06 | measured |
| pixel-11-pro | gpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the GPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 23485.4 | 3916.0095 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-07 | measured |
| pixel-11-pro | gpu | fallback | no | pass | sig:ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU | - | - | - | - | 3901.3949 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-06 | measured |
| pixel-11-pro | gpu | fallback | no | - | 1024 | - | 171.5131 | 7.6361 | 6102.2 | 584.6876 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-07 | measured |

## Pitfalls

- Keep the audio encoder on the CPU (audio_backend CPU, `--audio-backend cpu`): the Mac's GPU delegate refuses several of its encoder's operations, among them GATHER_ND and 5-dimensional tensors, and creating a conversation then fails with INTERNAL. The language model can run on the CPU or the GPU (card L67, L100).
- Context is 4,096 tokens, prompt and reply together: one message fits about 234 s of audio at the highest reply length of the card's 220 test clips, 264 s at their 95th percentile and 307 s at their median; a message that does not fit is refused with INVALID_ARGUMENT, not cut (a 421.6-second clip came to 4,236 tokens) (card L65, L198).
- Slower than granite-speech-5.0-2b_ekv2048.litertlm: on a Galaxy S26 the 9.9-second clip reached the first text in 3.83 s against 1.21 s (both with the language model on the GPU, the same session); the card keeps this file for messages of more than about 2 minutes of audio (card L37-L42, L211).
- One message holds one audio item, then the instruction as text (for example `transcribe the speech with proper punctuation and capitalization.`); send 16 kHz mono WAV (16-bit PCM and 32-bit float were tested). The encoder always computes a whole 36-second window, also for a short clip (card L63-L64, L195).
- Use a new conversation for each clip: from the second message of a conversation on, LiteRT-LM 0.19.1 puts the previous answer's stop token into the context with a zero embedding; not measured (card L102).
- With the language model on the CPU, LiteRT-LM quantizes the activations to int8 at run time; a second CPU process can print a slightly different transcript of the same clip (in the card's check of this file, two more commas) (card L197).
- The phone figures of 2026-11-06 and of 2026-11-07 differ (first text 4.36 s and 7.47 s on 2026-11-06, 5.16 s and 3.83 s on 2026-11-07, language model on the CPU and the GPU); the phone's throttling differed between the two days and the card did not separate the cause (card L155).

---

_Rendered by `edge-card` from `card.json` (the source of truth); edit the inputs, not this file._
