Foundation Models with a fallback for iPhones without Apple AI

The Foundation Models framework in iOS 26 gives every app access to Apple's on-device language model. No download, no API key, no server. The catch is the device list: Apple Intelligence runs on iPhone 15 Pro, iPhone 15 Pro Max and newer, only with iOS 26, only when the person has turned it on, and only once the system has finished downloading its model. A large share of the iPhones your app runs on today will report the model as unavailable.

If the feature you are building is central to the app, "not on your phone" is not an acceptable answer. This guide builds a three-tier chain: Apple's model first, a local GGUF model second, and your own rules as the floor. All three return the same Codable type.

Check availability, and say why

SystemLanguageModel.default.availability tells you whether the model can be used right now, and if not, why. The reasons matter for your UI: a device that is not eligible will never get it, while a device where the model is still downloading will have it in a few minutes.

AppleModel.swift

import FoundationModels

enum AppleModel {
    static var isAvailable: Bool {
        if #available(iOS 26.0, macOS 26.0, *),
           case .available = SystemLanguageModel.default.availability {
            return true
        }
        return false
    }

    /// Something you can show in Settings, or nil when the model is ready.
    static var unavailableReason: String? {
        if #available(iOS 26.0, macOS 26.0, *) {
            switch SystemLanguageModel.default.availability {
            case .available:
                return nil
            case .unavailable(.deviceNotEligible):
                return "This device doesn't support Apple Intelligence."
            case .unavailable(.appleIntelligenceNotEnabled):
                return "Apple Intelligence is turned off in Settings."
            case .unavailable(.modelNotReady):
                return "Apple Intelligence is still downloading its model."
            case .unavailable:
                return "Apple Intelligence isn't available right now."
            }
        }
        return "Apple Intelligence needs iOS 26."
    }
}

Keep the trailing case .unavailable:. The reason enum can grow in future iOS versions, and a catch-all keeps your switch compiling. If your deployment target is below iOS 26, the #available check is what lets the same binary run on iOS 17.

@Generable or DynamicGenerationSchema

Foundation Models offers two ways to get structured output. The familiar one is a macro on a Swift type:

Swift

import FoundationModels

@Generable
struct Letter {
    @Guide(description: "Who sent it, short")
    var sender: String
    @Guide(.anyOf(["none", "pay", "reply", "sign"]))
    var action: String
    @Guide(description: "Tiny steps, each starting with a verb", .maximumCount(4))
    var steps: [String]
}

let session = LanguageModelSession(instructions: "Read the letter. Use only facts from it.")
let letter = try await session.respond(to: scannedText, generating: Letter.self).content

That is the right choice when Apple's model is the only model you use. It stops being the right choice once a second tier is involved. The @Generable type exists only for Apple's framework, it only compiles where FoundationModels is available, and a local llama.cpp model knows nothing about it. You would end up describing the same shape twice: once as a macro, once as a grammar or prompt for the other model.

DynamicGenerationSchema builds the same constraint from data at runtime. If you already describe your output as a value (a list of fields with types and descriptions), you can turn that one description into a Foundation Models schema and into a GBNF grammar for llama.cpp. One description, two engines.

Describe the answer once

Here is a minimal field description, the same idea the kit's Schema type uses:

Fields.swift

struct Property {
    let name: String
    let field: Field
    let description: String?
}

indirect enum Field {
    case string
    case choice([String])
    case integer(ClosedRange<Int>?)
    case boolean
    case date                      // "YYYY-MM-DD" or ""
    case list(Field, max: Int?)
}

let letterFields: [Property] = [
    .init(name: "sender", field: .string, description: "Who sent it, short"),
    .init(name: "action", field: .choice(["none", "pay", "reply", "sign"]), description: nil),
    .init(name: "urgency", field: .integer(0...3), description: "0 information only, 3 deadline soon"),
    .init(name: "deadline", field: .date, description: "Only if the letter names one"),
    .init(name: "steps", field: .list(.string, max: 4), description: "Tiny steps, each starting with a verb"),
]

Turn it into a DynamicGenerationSchema

Each field type maps to one initializer. Choices become anyOf, ranges become a .range guide, a date becomes a string with a regular-expression guide, and lists carry their maximum length.

AppleSchema.swift

import FoundationModels

@available(iOS 26.0, macOS 26.0, *)
extension Field {
    func dynamic(name: String) -> DynamicGenerationSchema {
        switch self {
        case .string:
            DynamicGenerationSchema(type: String.self)
        case .choice(let values):
            DynamicGenerationSchema(name: name, anyOf: values)
        case .integer(let range):
            DynamicGenerationSchema(type: Int.self, guides: range.map { [.range($0)] } ?? [])
        case .boolean:
            DynamicGenerationSchema(type: Bool.self)
        case .date:
            DynamicGenerationSchema(type: String.self, guides: [.pattern(/^$|^\d{4}-\d{2}-\d{2}$/)])
        case .list(let item, let max):
            DynamicGenerationSchema(arrayOf: item.dynamic(name: name + "_item"), maximumElements: max)
        }
    }
}

@available(iOS 26.0, macOS 26.0, *)
func appleJSON(schemaName: String, fields: [Property], instructions: String, input: String) async throws -> String {
    let root = DynamicGenerationSchema(
        name: schemaName,
        description: nil,
        properties: fields.map { p in
            DynamicGenerationSchema.Property(
                name: p.name,
                description: p.description,
                schema: p.field.dynamic(name: "\(schemaName)_\(p.name)")
            )
        }
    )
    let schema = try GenerationSchema(root: root, dependencies: [])
    let session = LanguageModelSession(instructions: instructions)
    let response = try await session.respond(to: input, schema: schema)
    return response.content.jsonString
}

Two details. Every anyOf schema needs a unique name, which is why the property name is folded into it. And response.content is a GeneratedContent, whose jsonString gives you plain JSON. Decode it with JSONDecoder into the same Codable struct every other tier returns:

Swift

struct LetterAnswer: Codable, Sendable {
    var sender: String
    var action: String
    var urgency: Int
    var deadline: String
    var steps: [String]
}

let json = try await appleJSON(schemaName: "Letter", fields: letterFields,
                               instructions: "Read the letter. Use only facts from it.",
                               input: scannedText)
let letter = try JSONDecoder().decode(LetterAnswer.self, from: Data(json.utf8))

The property order of the struct does not matter to JSONDecoder, but the order of properties does matter to the model: it writes the keys in that order. Put the fields that help it think (sender, action) before the ones that depend on them (steps).

The fallback chain

With Apple's tier returning the same JSON as everything else, the chain is a loop over tiers that returns the first answer that decodes:

Assistant.swift

enum Tier: String { case apple, local, rules }

struct LetterAssistant {
    var localModel: () async -> URL?                  // nil until the GGUF is downloaded
    var local: (URL, String) async throws -> LetterAnswer
    var rules: (String) -> LetterAnswer?

    func run(_ input: String) async throws -> (LetterAnswer, Tier) {
        var lastError: Error = CancellationError()
        for tier in [Tier.apple, .local, .rules] {
            do {
                switch tier {
                case .apple:
                    guard AppleModel.isAvailable, #available(iOS 26.0, macOS 26.0, *) else { continue }
                    let json = try await appleJSON(schemaName: "Letter", fields: letterFields,
                                                   instructions: "Read the letter. Use only facts from it.",
                                                   input: input)
                    return (try JSONDecoder().decode(LetterAnswer.self, from: Data(json.utf8)), .apple)
                case .local:
                    guard let url = await localModel() else { continue }
                    return (try await local(url, input), .local)
                case .rules:
                    guard let answer = rules(input) else { continue }
                    return (answer, .rules)
                }
            } catch {
                lastError = error   // try the next tier
            }
        }
        throw lastError
    }
}

Catching per tier is the important part. Apple's model can throw for reasons you do not control: the guardrails reject an input, the context window is exceeded, the system is under memory pressure. When that happens the next tier simply gets its turn.

The local closure is where a llama.cpp model goes. Running a GGUF model on iPhone covers loading and generating; a grammar built from the same letterFields guarantees the output decodes into LetterAnswer. The rules closure is plain Swift: keywords, regular expressions, NSDataDetector for dates and amounts. It will not write good steps, but it can fill action and deadline well enough that the feature still works offline on an iPhone 11.

Which device gets which tier

Device	Tier you can expect
iPhone 15 Pro and newer, iOS 26, Apple Intelligence on	Apple
Same devices with Apple Intelligence off, or its model still downloading	Local, if downloaded; otherwise rules
iPhone 13 Pro / Pro Max, iPhone 14, iPhone 15 and 15 Plus (6 GB RAM)	Local 2–3B model, 4-bit
iPhone 13 / 13 mini, iPhone 12 and older (4 GB RAM or less)	Rules, or a 1B model for short tasks

A 4-bit 2–3B model needs about 1.6–2 GB of RAM for its weights plus the context. That is comfortable on 6 GB devices and risky on 4 GB ones, where the system will terminate your app before llama.cpp reports an error. Gate the local tier on ProcessInfo.processInfo.physicalMemory rather than on model names.

Prompting two different models

Apple's model and a 2B open model respond differently to the same instructions. A few habits keep their answers close:

Keep instructions short and literal. "Use only facts from the letter" works for both; long rule lists confuse the smaller model.
Give one worked example. For Apple's model, put it in the prompt text ("Example input … Example answer …"). For llama.cpp, add it as a user/assistant pair in the chat. One good example does more than a page of rules.
Put the meaning of each field in its description. Both engines see it: Apple's through the schema, the local model through a short key list in the system prompt.
Show the tier in debug builds. When an answer looks wrong, you want to know which model wrote it.