---
family: granite-speech
license: apache-2.0
model_id: granite-speech-5.0-2b-ekv2048
source_url: https://huggingface.co/litert-community/granite-speech-5.0-2b
task: automatic-speech-recognition
---

# granite-speech-5.0-2b-ekv2048

| | |
|---|---|
| **Task** | automatic-speech-recognition |
| **Family** | granite-speech |
| **Source** | https://huggingface.co/litert-community/granite-speech-5.0-2b |
| **License** | apache-2.0 |

## Conversion

| | |
|---|---|
| **Tool** | litert-torch 0.10.0 (export_hf for the Granite language model, the token embedding exported separately; the audio encoder as a PyTorch module traced with litert-torch) packed with litert-lm-builder 0.19.1; repro = hf-to-litertlm REPRODUCE.md, the granite-speech-5.0-2b_ekv2048.litertlm entry (card L204-L207) litert-torch 0.10.0, litert-lm-builder 0.19.1 (litert-torch 0.10.0's own packing calls a builder API that 0.19 no longer has); LiteRT-LM 0.19.1 for every measurement except the encoder-alone times (ai-edge-litert 2.3.0) (card L31, L203, L206) |
| **Command** | `ibm-granite/granite-speech-5.0-2b rev 98160169 -> the language model: export_hf, prefill signatures 64, 256 and 1,024, recipe dynamic_wi8_afp32, KV cache 2,048; the audio encoder: written with tensors of rank 4 or less (the relative-position shift as a skew, the convolutions as 2-D convolutions, no GATHER_ND, BROADCAST_TO or DIV), exported for a 24-second window, float16 weights; the source's tokenizer.json byte for byte, a chat template rendering `USER: <|audio|>{instruction}\n ASSISTANT:`, stop token <|end_of_text|> (100257), model type generic_model (card L59, L202, L204-L205)` |
| **Quantization** | language model: int8 weights (one scale per output channel), KV cache of 2,048 tokens, no activation-type declaration (float16 activations on the GPU, the runtime's default); token embedding table int8; audio encoder: float16 weights computed in float32, one 24-second window (card L48, L54-L56) |

## Artifacts

| File | SHA-256 | Size (MB) |
|---|---|---|
| `granite-speech-5.0-2b_ekv2048.litertlm` | `bf8eb5a8075b4fd3cdb1109a9edf0c247ff2bfb52b95699c31626aae80f3a642` | 2693.554 |

## Performance

No benchmark data yet.

## Device runs (NPU / LiteRT-LM)

Model-level on-device results — measured behavior of this exact artifact on the recorded device, only meaningful together with that environment. Throughput is conditional on the measured prompt length (Prompt (tokens); '-' = the source stated none): rows differing only there are different measurements, not re-runs. Full records (failure classes, log evidence): `data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b-ekv2048__galaxy-s26.json`, `data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b-ekv2048__mac-studio-m4-max.json`, `data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b-ekv2048__pixel-11-pro.json`.

| Device | Accelerator | Status | Full delegation | Output match | Prompt (tokens) | Latency p50 (ms) | Prefill tok/s | Decode tok/s | TTFT (ms) | Peak mem (MB) | Environment | Date | Provenance |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU at the default thread count, in the thread-count session of 2026-11-07 09:10-09:18 JST (the default and thread_count=6 alternating, 2 processes each), audio encoder on the GPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1632.55 | 4305.4162 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU at thread_count=6, audio encoder on the GPU, in the session of 2026-11-07 15:15-15:33 JST (this file and an unpublished build with the audio encoder on the NPU alternating per language-model backend, A B B A, 2 processes each); ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1615.0 | 4346.9373 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU at thread_count=6, in the thread-count session of 2026-11-07 09:10-09:18 JST (the default and thread_count=6 alternating, 2 processes each), audio encoder on the GPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1434.35 | 4305.0353 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU at thread_count=6, in the thread-count session of 2026-11-07 15:01-15:10 JST (thread_count=6 and 8 alternating, 2 processes each), audio encoder on the GPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1606.5 | 4305.6128 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU at thread_count=8, in the thread-count session of 2026-11-07 15:01-15:10 JST (thread_count=6 and 8 alternating, 2 processes each), audio encoder on the GPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1565.2 | 4305.5063 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU, audio encoder on the GPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1684.75 | 4308.3653 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | pass | sig:the source card's multilingual_sample.wav (24.9 s, English and then French) with the instruction `transcribe the speech with proper punctuation and capitalization.`, one call without streaming (then a text-only control), greedy, at most 256 new tokens; output_match = the reply stops, is not empty and is within 10 % word error rate of the source model's transcript (the line, set before the runs); audio encoder on the GPU | - | - | - | - | 4509.1881 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | cpu | fallback | no | - | 1024 | - | 229.1739 | 15.3363 | 4539.75 | 2630.7953 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | gpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the GPU, audio encoder on the GPU, in the session of 2026-11-07 15:15-15:33 JST (this file and an unpublished build with the audio encoder on the NPU alternating per language-model backend, A B B A, 2 processes each); ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1319.25 | 2569.9656 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | gpu | fallback | no | - | sig:ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the GPU, audio encoder on the GPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included | - | - | - | 1211.9 | 2545.3486 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| galaxy-s26 | gpu | fallback | no | pass | sig:the source card's multilingual_sample.wav (24.9 s, English and then French) with the instruction `transcribe the speech with proper punctuation and capitalization.`, one call without streaming (then a text-only control), greedy, at most 256 new tokens; output_match = the reply stops, is not empty and is within 10 % word error rate of the source model's transcript (the line, set before the runs); audio encoder on the GPU | - | - | - | - | 2533.7037 | Galaxy S26 (SM-S942Q) · Qualcomm SM8850 (Adreno GPU) · litert-lm 0.19.1 · Android 16 (BP4A.251205.006) | 2026-11-07 | measured |
| mac-studio-m4-max | cpu | pass | - | - | sig:ASR clip timing on the Mac: a warm-up clip, then a 5.9 s LibriSpeech clip, a 10.0 s and a 29.3 s FLEURS en_us clip in that order, one streaming call per clip with a new conversation each, the model loaded once per process, the transcription instruction, greedy, at most 256 new tokens; the median of 5 processes; audio encoder on the GPU; ttft_ms = the 10.0 s clip's (the third conversation's) first streamed text, audio encoding included | - | - | - | 409.7 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-07 | measured |
| mac-studio-m4-max | cpu | pass | - | pass | sig:ASR gate: the 220 clips (20 LibriSpeech dev-clean, 50 FLEURS en_us, 50 FLEURS ja_jp and 25 FLEURS clips each of French, German, Spanish and Portuguese), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; scored against the source model in transformers 5.22.1 on the CPU (English with Whisper's English normalizer, the other European languages with its basic normalizer, Japanese by characters); line = the source model + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the GPU | - | - | - | - | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-07 | measured |
| mac-studio-m4-max | cpu | pass | - | - | 1024 | - | 841.62 | 36.21 | 1245.2 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-07 | measured |
| mac-studio-m4-max | gpu | pass | - | - | sig:ASR clip timing on the Mac: a warm-up clip, then a 5.9 s LibriSpeech clip, a 10.0 s and a 29.3 s FLEURS en_us clip in that order, one streaming call per clip with a new conversation each, the model loaded once per process, the transcription instruction, greedy, at most 256 new tokens; the median of 5 processes; audio encoder on the GPU; ttft_ms = the 10.0 s clip's (the third conversation's) first streamed text, audio encoding included | - | - | - | 225.8 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-07 | measured |
| mac-studio-m4-max | gpu | pass | - | pass | sig:ASR gate: the 220 clips (20 LibriSpeech dev-clean, 50 FLEURS en_us, 50 FLEURS ja_jp and 25 FLEURS clips each of French, German, Spanish and Portuguese), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; scored against the source model in transformers 5.22.1 on the CPU (English with Whisper's English normalizer, the other European languages with its basic normalizer, Japanese by characters); line = the source model + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the GPU | - | - | - | - | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-07 | measured |
| mac-studio-m4-max | gpu | pass | - | - | 1024 | - | 3373.13 | 109.87 | 312.7 | - | Mac Studio (M4 Max, Mac16,9) · Apple M4 Max · litert-lm 0.19.1 · macOS 27.0 (26A428) | 2026-11-07 | measured |
| pixel-11-pro | cpu | fallback | no | pass | sig:ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the GPU | - | - | - | - | 2633.9615 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-07 | measured |
| pixel-11-pro | gpu | fallback | no | pass | sig:ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the GPU | - | - | - | - | 1474.4863 | Pixel 11 Pro · Google Tensor G6 · litert-lm 0.19.1 · Android 17 (CD1A.260905.001.B1) | 2026-11-07 | measured |

## Pitfalls

- Context is 2,048 tokens, prompt and reply together: one message fits about 116 s of audio at the highest reply length of the card's 220 test clips, 131 s at their 95th percentile and 152 s at their median; a message that does not fit is refused with INVALID_ARGUMENT, not cut. Take granite-speech-5.0-2b.litertlm (4,096 tokens) when one message has to hold more than about 2 minutes of audio (card L42, L65, L198).
- One message holds one audio item, then the instruction as text (for example `transcribe the speech with proper punctuation and capitalization.`); send 16 kHz mono WAV (16-bit PCM and 32-bit float were tested; other sample rates, stereo and other containers were not) (card L63-L64).
- The audio encoder runs on the GPU (audio_backend GPU, `--audio-backend gpu`); the card did not test this file with the audio encoder on the CPU. The encoder always computes a whole 24-second window, also for a short clip (card L64, L67, L195).
- Use a new conversation for each clip: from the second message of a conversation on, LiteRT-LM 0.19.1 puts the previous answer's stop token into the context with a zero embedding, so a second transcript in one conversation would start from a context the model never saw; not measured (card L102).
- A file that ends in exact digital silence is read as slightly shorter (the encoder takes the last non-zero sample as the end); on the card's 5 test files with 173 to 274 trailing zero samples the source language model's transcript was unchanged (card L66).
- With the language model on the CPU, LiteRT-LM quantizes the activations to int8 at run time: the transcripts were the source model's character for character on 180 of 220 clips against 195 on the GPU, at error rates within 0.3 points of each other (card L197).
- Galaxy S26 CPU threads: Backend.CPU(thread_count=6) reached the first text sooner than the default (4 in the LiteRT-LM 0.19.1 source): 1.43 s against 1.63 s on the 9.9-second clip in one session; the LM benchmark was not faster with 6 threads (card L100, L117).
- No NPU file and no int4 file: an audio encoder compiled for the Galaxy S26 NPU turned the English part of the source card's sample into French (the NPU's sigmoid returns 0 below about -5.7); the fixed encoder was slower than this file's GPU window on clips up to 24 s; an int4 language model prefilled at 42 % of int8's rate on a Mac CPU (card L196).

---

_Rendered by `edge-card` from `card.json` (the source of truth); edit the inputs, not this file._
