Define the task before choosing a model
For offline chat, validate text input and output first. Add image or audio input only after checking support for that model and the current llama.cpp build. A multimodal model card alone does not prove that the iOS sample supports every modality.
The source notes discuss Qwen3.5-0.8B and Gemma 4 E2B and contain subjective device observations. The configurations are not sufficiently standardized for a general token-per-second ranking, so we do not publish one here.
Prepare the sample and GGUF file
On a Mac, prepare Xcode, device signing, enough storage, and a GGUF from a trusted publisher. Verify license, file integrity, quantization and compatibility with the llama.cpp revision you use. Follow the linked SwiftUI README if repository commands or project paths change.
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
./build-xcframework.sh
open examples/llama.swiftui/llama.swiftui.xcodeproj- Build the framework from the repository root, then open the SwiftUI sample in Xcode.
- Set your own signing team and target device.
- Give the sample a model you are licensed to use, then validate a short single-turn text prompt.
- Record the repository revision, model file and quantization, iPhone model and iOS version before changing parameters.
When a downloaded model cannot run
Separate disk size from runtime use: model weights, context and KV cache, optional mmproj assets for some vision models, temporary buffers, and other apps all compete for memory. A file fitting on disk is not proof that it can run reliably.
Reproduce with a short text-only context, then increase context length or enable vision one change at a time. Preserve Xcode logs and measured peak memory when the app is terminated or stalls.
Swipe the table sideways to see every column →
| Symptom | Check first | Next step |
|---|---|---|
| Model fails to load | GGUF, quantization and runtime compatibility | Return to a supported sample model |
| Short prompts work, long ones fail | Context length and KV cache | Reduce context and measure peak memory |
| Text works, images fail | Model ability, mmproj and multimodal runtime | Check mtmd documentation |
| Speed varies | First token, sustained output, heat and background load | Repeat a fixed test |
Measure useful performance, not one headline number
Measure time to first token separately from sustained tokens per second. Fix prompt length, generated length, context, quantization, acceleration and device state; repeat tests and report a range. Test a longer conversation too.
The original speed figures come from a particular device and setup. We have not rerun them and do not extrapolate to other iPhones. Task quality, failure rate, peak memory and temperature matter alongside throughput.
Ship with clear limits
Keep model download, updates and data permissions separate from inference. Tell users the model size, offline behavior and whether any other part of the app sends data out. Local inference alone does not prove that an entire app makes no network requests. Provide a recoverable error on memory pressure.
Sources checked November 6, 2026. The sample and multimodal interfaces evolve; verify the linked repository documentation before integrating them into a product.