Run an LLM locally on iPhone with Swift and llama.cpp (2026)

Running a model on the device means no server bill, no API key in your binary and no user text leaving the phone. The price is memory, a large download and a C API. The memory and the download are covered below and in the background download guide. The C API is smaller than it looks: about a dozen functions get you from a file on disk to streamed text.

What you need

Xcode 26 and Swift 6. The code below also works with Swift 5 language mode.
A real iPhone or iPad, or your Mac. The official llama.cpp xcframework has slices for iOS devices and macOS, but no iOS Simulator slice. If you need the Simulator, build the framework yourself with build-xcframework.sh from the llama.cpp repository.
A GGUF model. A 4-bit quantised 2–3B model is the practical size for phones. LFM2 2.6B in Q4_K_M is 1.6 GB.
Enough RAM. A 1.6 GB model wants a device with 6 GB: iPhone 13 Pro and Pro Max, iPhone 14 and newer. The iPhone 13 and 13 mini have 4 GB, which is tight for anything above 1B parameters.

Add llama.cpp with Swift Package Manager

llama.cpp publishes a prebuilt xcframework with every release. Reference it as a binary target, pinned to a release and its checksum, so builds stay reproducible:

Package.swift

// swift-tools-version: 6.0
import PackageDescription

let package = Package(
    name: "LocalModel",
    platforms: [.iOS(.v17), .macOS(.v14)],
    products: [.library(name: "LocalModel", targets: ["LocalModel"])],
    targets: [
        // llama.cpp b11887, official release build (MIT). iPhone (arm64) and Mac.
        .binaryTarget(
            name: "llama",
            url: "https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-xcframework.zip",
            checksum: "85aa7c9195918470c2de8dadf432bd8d01f5f7de2a4e65c1014491880ce673b6"
        ),
        .target(name: "LocalModel", dependencies: ["llama"]),
    ]
)

The official xcframework has slices for iOS devices and macOS, but none for the iOS Simulator, so an app that links it won’t build for the Simulator. If you need that, use an unchanged build of the same release with a Simulator slice, or build one yourself with build-xcframework.sh from the llama.cpp repository. In the Simulator, load the model with n_gpu_layers = 0: its Metal lacks most of llama.cpp’s kernels and ends up slower than the CPU.

When you move to a newer release, update the URL and get the new checksum with swift package compute-checksum llama-bXXXX-xcframework.zip. The C API changes between releases now and then, so pin it and upgrade on purpose.

Load the model

Loading takes two steps: the model (weights) and a context (the working memory for one conversation). Wrap both in an actor so generation never runs on the main thread, and keep the raw pointers in a small class that frees them exactly once.

LLM.swift

import Foundation
import llama

enum LLMError: Error {
    case loadFailed
    case promptTooLong(tokens: Int, limit: Int)
    case decodeFailed
}

/// Owns the llama.cpp pointers and frees them when the last reference goes away.
private final class Handle: @unchecked Sendable {
    let model: OpaquePointer
    let ctx: OpaquePointer
    let vocab: OpaquePointer
    init(model: OpaquePointer, ctx: OpaquePointer) {
        self.model = model
        self.ctx = ctx
        vocab = llama_model_get_vocab(model)
    }
    deinit {
        llama_free(ctx)
        llama_model_free(model)
    }
}

actor LLM {
    private let handle: Handle
    private let template: String?
    let contextLength: Int32 = 4096

    init(modelURL: URL) throws {
        llama_backend_init()

        var mp = llama_model_default_params()
        mp.n_gpu_layers = 99                       // all layers on the GPU (Metal)
        guard let model = llama_model_load_from_file(modelURL.path, mp) else { throw LLMError.loadFailed }

        var cp = llama_context_default_params()
        cp.n_ctx = UInt32(contextLength)           // prompt + answer
        cp.n_batch = cp.n_ctx                      // the whole prompt fits in one llama_decode call
        cp.n_ubatch = 512
        let threads = Int32(max(2, ProcessInfo.processInfo.activeProcessorCount - 2))
        cp.n_threads = threads
        cp.n_threads_batch = threads

        guard let ctx = llama_init_from_model(model, cp) else {
            llama_model_free(model)
            throw LLMError.loadFailed
        }
        handle = Handle(model: model, ctx: ctx)
        template = llama_model_chat_template(model, nil).map { String(cString: $0) }
    }
}

Two settings matter more than the rest. n_gpu_layers = 99 puts every layer on Metal; on Apple silicon the GPU shares memory with the CPU, so there is no copy cost. n_batch = n_ctx prevents a crash with long prompts that most first integrations run into. The n_batch guide explains why.

Loading a 1.6 GB file is not instant. On an M-series Mac the first load of LFM2 2.6B took around 20–30 seconds in our measurements, later loads are faster because the file is in the page cache. Load once, keep the actor alive, and release it on a memory warning.

Build the prompt with the model's chat template

Every instruction-tuned model expects its own prompt format: special tokens for roles, turn ends and where the answer starts. GGUF files carry that template, and llama.cpp can apply it. Use it instead of hard-coding a format, and the same code works for LFM2, Qwen, Gemma, Llama and Phi.

Swift

extension LLM {
    func prompt(_ messages: [(role: String, content: String)]) -> String {
        let roles = messages.map { strdup($0.role)! }
        let contents = messages.map { strdup($0.content)! }
        defer { (roles + contents).forEach { free($0) } }
        var chat = zip(roles, contents).map { llama_chat_message(role: $0, content: $1) }

        var buf = [CChar](repeating: 0, count: 16_384)
        var n = llama_chat_apply_template(template, &chat, chat.count, true, &buf, Int32(buf.count))
        if n > buf.count {                          // too small: the return value is the size needed
            buf = [CChar](repeating: 0, count: Int(n) + 1)
            n = llama_chat_apply_template(template, &chat, chat.count, true, &buf, Int32(buf.count))
        }
        if n > 0 { return String(decoding: buf.prefix(Int(n)).map { UInt8(bitPattern: $0) }, as: UTF8.self) }

        // Unknown template: ChatML is the most common format.
        return messages.map { "<|im_start|>\($0.role)\n\($0.content)<|im_end|>\n" }.joined() + "<|im_start|>assistant\n"
    }
}

The true argument appends the assistant prefix, so the model starts writing the answer instead of continuing the user's message.

Tokenize, decode, sample

Generation is a loop: turn the prompt into tokens, feed them to the model once, then sample one token at a time and feed each back in until the model emits an end-of-generation token.

Swift

extension LLM {
    func generate(_ messages: [(role: String, content: String)], maxTokens: Int = 512) throws -> String {
        let ctx = handle.ctx, vocab = handle.vocab
        llama_memory_clear(llama_get_memory(ctx), true)     // start a fresh conversation

        var tokens = tokenize(prompt(messages))
        let limit = Int(contextLength) - maxTokens
        guard tokens.count <= limit else { throw LLMError.promptTooLong(tokens: tokens.count, limit: limit) }

        guard let sampler = llama_sampler_chain_init(llama_sampler_chain_default_params()) else { throw LLMError.decodeFailed }
        defer { llama_sampler_free(sampler) }
        llama_sampler_chain_add(sampler, llama_sampler_init_greedy())

        let ok = tokens.withUnsafeMutableBufferPointer {
            llama_decode(ctx, llama_batch_get_one($0.baseAddress, Int32($0.count))) == 0
        }
        guard ok else { throw LLMError.decodeFailed }

        var bytes: [UInt8] = []
        for _ in 0..<maxTokens {
            var token = llama_sampler_sample(sampler, ctx, -1)
            if llama_vocab_is_eog(vocab, token) { break }
            bytes += piece(token)
            let next = withUnsafeMutablePointer(to: &token) { llama_decode(ctx, llama_batch_get_one($0, 1)) }
            if next != 0 { break }
        }
        return String(decoding: bytes, as: UTF8.self)
    }

    private func tokenize(_ text: String) -> [llama_token] {
        let vocab = handle.vocab
        let len = Int32(text.utf8.count)
        let needed = -llama_tokenize(vocab, text, len, nil, 0, false, true)   // parse special tokens
        guard needed > 0 else { return [] }
        var tokens = [llama_token](repeating: 0, count: Int(needed))
        let n = llama_tokenize(vocab, text, len, &tokens, needed, false, true)
        return n > 0 ? Array(tokens.prefix(Int(n))) : []
    }

    private func piece(_ token: llama_token) -> [UInt8] {
        var buf = [CChar](repeating: 0, count: 64)
        var n = llama_token_to_piece(handle.vocab, token, &buf, Int32(buf.count), 0, false)
        if n < 0 {
            buf = [CChar](repeating: 0, count: Int(-n))
            n = llama_token_to_piece(handle.vocab, token, &buf, Int32(buf.count), 0, false)
        }
        return n > 0 ? buf.prefix(Int(n)).map { UInt8(bitPattern: $0) } : []
    }
}

A few details are easy to get wrong:

Tokenize with special tokens on. The chat template contains markers like <|im_start|>. With parse_special = false they become ordinary text and the model gets confused.
Collect bytes, not strings. A token can end in the middle of a multi-byte character such as "ä" or "漢". Convert the bytes once at the end, or, when streaming, hold bytes back until String(bytes:encoding: .utf8) succeeds.
Check the length first. Throwing promptTooLong before decoding is far better than letting llama.cpp fail halfway.
Greedy sampling for extraction. For tasks with one right answer, temperature 0 (greedy) gives stable output. For chat, swap in llama_sampler_init_top_p, llama_sampler_init_temp and llama_sampler_init_dist.

Use it from SwiftUI

Swift

let llm = try LLM(modelURL: modelFile)
let reply = try await llm.generate([
    (role: "system", content: "Answer in one sentence."),
    (role: "user", content: "Why is the sky blue?"),
])

Because LLM is an actor, the call suspends and the UI stays responsive. Show a progress state during the first load and keep the instance in a long-lived object, not in a view that gets recreated.

Memory and the App Store

The model's weights, the KV cache for the context and Metal buffers all count against your app's memory limit. Three things help:

Add the com.apple.developer.kernel.increased-memory-limit entitlement. It raises the limit on devices that allow it.
Keep one model loaded. Two 1.6 GB models rarely fit on a phone at the same time, so free the old one before loading another.
Listen for memory warnings and drop the model. Loading it again is slower than generating, but far better than being terminated.

Do not bundle the model in the app. Download it on request, show its size first and verify it after the download; this guide covers a background download with mirrors and SHA-256. Mention the download in your App Review notes and link the model's license in your settings; see the App Review notes in the docs.

Next: structured answers

Free text is fine for chat. Most apps need fields: a date, an amount, a category. A small model asked politely for JSON will sometimes forget a brace. A grammar makes broken JSON impossible; the GBNF guide builds one from a Swift description. If you also want Apple Intelligence on devices that have it, the Foundation Models fallback guide shows how to put both behind one call.

What you need

Add llama.cpp with Swift Package Manager

Load the model

Build the prompt with the model's chat template

Tokenize, decode, sample

Use it from SwiftUI

Memory and the App Store

Next: structured answers

Related guides