OnDevice Kit

Any LLM feature.
On every iPhone.

Pull data out of text, sort and tag, summarize, draft replies, turn voice notes into tasks, chat. OnDevice Kit runs these LLM features inside your iOS app: no server, no API bill, nothing leaves the phone.

Apple Intelligence answers where it exists, a local model via llama.cpp where it doesn’t, your own rules as the floor. You describe the answer once and get the same Codable value, whatever answered.

Swift 6, iOS 17+ and macOS 14+. One-time purchase, full source, extracted from two of our own apps.

Try a use case
LetterReader.swift
import OnDeviceKit

struct Letter: Codable, Sendable {
    var sender: String
    var action: String
    var deadline: String
    var steps: [String]
}

let schema = Schema("Letter") {
    Property("sender", .string, "Who sent it, short")
    Property("action", .choice(["none", "pay", "reply", "sign"]))
    Property("deadline", .date, "Only if the letter names one")
    Property("steps", .list(.string, max: 4), "Tiny steps")
}

// Downloads once, in the background, checked against its SHA-256
let store = ModelStore(.lfm2_2_6B)

let assistant = Assistant<Letter>(
    schema: schema,
    instructions: "Read the letter. Use only facts from it.",
    model: store.readyFile,
    rules: { text in Letter.guess(from: text) }
)

let (letter, tier) = try await assistant.run(scannedText)
// tier: .apple, .local or .rules
Scanned letter

City Energy, second reminder. We have not received payment for invoice 4711. Please transfer 84.20 EUR by 30 October 2026, reference 4711, or late fees apply.

Run it on
  1. Apple Intelligence Foundation Models, iOS 26 skipped: device not eligible
  2. Local GGUF model llama.cpp on Metal, GBNF grammar answered
  3. Your rules keywords, regular expressions not needed
letter, tier: .local
{
  "sender": "City Energy",
  "action": "pay",
  "deadline": "2026-10-30",
  "steps": [
    "Transfer 84.20 EUR",
    "Use reference 4711",
    "Pay before 30 October"
  ]
}

An illustration: real wording depends on the model and your instructions; the shape never does.

What you can build

If a feature can be described as “text in, answer out”, it runs on the device. Describe the answer as a Schema and the kit makes every tier return exactly that, or stream free text when you don’t need structure.

  • Read documents

    Letters, invoices, contracts, receipts from the camera: sender, amounts, dates, what to do next.

    .date, .number, .choice
  • Sort and tag

    Route messages, label notes, detect the topic and mood of feedback or journal entries.

    .choice, .list(.string)
  • Voice to structure

    Speech framework transcript in, tasks, shopping lists or step-by-step guides out.

    .list(.object(Task))
  • Summarize

    Two calm sentences for a long email, a chat thread or a page of notes.

    .string
  • Draft replies

    A first answer to a customer, a polite “no”, three short reply suggestions.

    .list(.string, max: 3)
  • Fill in forms

    Paste an address, a booking confirmation or a CV and fill the fields of your form.

    .object(Address)
  • Rewrite

    Plain language, a friendlier tone, shorter, or another language for short texts.

    .string
  • Chat

    A private assistant that streams its answer token by token, with any GGUF model and its own chat template.

    LocalLLM.stream

Where small on-device models struggle: very long documents (a few pages is fine, a book is not), knowledge about the world they weren’t given, and long chains of reasoning. Give them the text, ask about that text, and they are surprisingly good.

What’s in the package

Four Swift modules, an example app, tests and a screenshot pipeline. The parts every on-device AI app needs and nobody wants to write twice.

Schema

OnDeviceLLM

Describe the JSON you want once. The kit compiles it to a GBNF grammar for llama.cpp, so decoding is constrained and a small model cannot produce broken JSON, and to a DynamicGenerationSchema for Apple’s Foundation Models.

Field types: .string, .choice, .integer(0...3), .number, .boolean, .date, .list(_, max:) and nested .object.

How GBNF forces valid JSON

schema.gbnf (excerpt)
root ::= "{" ws "\"sender\":" ws str "," ws "\"action\":" ws root-action-choice "," ws "\"deadline\":" ws date "," ws "\"steps\":" ws root-steps-list ws "}"
root-action-choice ::= "\"none\"" | "\"pay\"" | "\"reply\"" | "\"sign\""
root-steps-list ::= "[" ws "]" | "[" ws str ( ws "," ws str ){0,3} ws "]"
date ::= "\"\"" | "\"" [0-9]{4} "-" [0-9]{2} "-" [0-9]{2} "\""
ws ::= [ \t\n]{0,8}

LocalLLM

OnDeviceLLM

  • Uses the chat template stored in the model file, so LFM2, Qwen, Gemma, Llama and Phi work without prompt fiddling.
  • Sets n_batch = n_ctx, so long prompts don’t hit llama.cpp’s GGML_ASSERT(n_tokens_all <= cparams.n_batch) abort.
  • Throws promptTooLong before generating instead of crashing.
  • Streams whole characters only: a token that ends inside “ä” or “漢” waits for the rest.

The n_batch crash, explained

LocalLLM
let llm = try LocalLLM(model: url)              // loads the GGUF once

// Free text
let summary = try await llm.generate([
    .system("Summarize in two calm sentences."), .user(email)
])

// Chat, streamed token by token
for try await piece in llm.stream(history + [.user(question)]) {
    answer += piece
}

// Structured, forced into your schema
let recipe = try await llm.generate(Recipe.self, schema: recipeSchema,
                                    instructions: "...", input: pastedText)

Assistant

OnDeviceLLM

Tries [.apple, .local, .rules] in order (configurable) and returns the first answer that decodes, together with the tier that produced it. Pass one or two Examples: a single good example teaches a 2B model more than a page of rules.

ModelCache.shared keeps one model in memory across calls; unload() frees it on a memory warning.

Foundation Models with a fallback

Assistant
let assistant = Assistant<Note>(
    schema: Note.schema,
    instructions: "Turn the text into a calm, clear note.",
    examples: [Note.example],
    order: [.apple, .local, .rules],
    model: store.readyFile,
    rules: Note.guess
)

if await assistant.currentTier() == nil { showDownloadPrompt = true }
let (note, tier) = try await assistant.run(input)

ModelStore

OnDeviceModels

  • Background URLSession: keeps downloading while the app is closed and reattaches on the next launch.
  • Tries each URL in order, your mirror first and Hugging Face second. HTTP errors and broken files move on to the next one.
  • Verifies SHA-256 off the main thread and excludes the file from iCloud backup.

Background downloads for large models

ModelStore
@State private var store = ModelStore(
    .lfm2_2_6B.mirrored(first: URL(string: "https://your.cdn/lfm2.gguf")!)
)

switch store.state {
case .notDownloaded:      Button("Download (1.6 GB)") { store.download() }
case .downloading(let p): ProgressView(value: p)
case .verifying:          Text("Checking …")
case .ready:              Text("Ready")
case .failed(let reason): Text(reason)
}

FreeQuota, DailyPass, Unlock

OnDeviceStore

A free quota kept in the Keychain, so reinstalling doesn’t reset it. A once-a-day pass. A StoreKit 2 non-consumable that listens to Transaction.updates, handles Ask to Buy, Family Sharing and refunds, and caches the last state so your UI doesn’t flash a paywall at launch.

A free trial that survives reinstalling

OnDeviceStore
let quota = FreeQuota("summaries", free: 3)          // Keychain
@State var unlock = Unlock("com.you.app.unlimited")  // StoreKit 2

if quota.isAvailable || unlock.isUnlocked {
    summarize(); quota.use()
} else {
    showPaywall = true
}
Button("Unlock for \(unlock.price ?? "…")") { Task { try await unlock.buy() } }
Button("Restore purchase") { Task { try? await unlock.restore() } }

App Store screenshots

Tools/screenshots

capture.sh builds your app, sets a clean 9:41 status bar and shoots every screen in every language in the Simulator. compose.py adds captions and a device frame in the two sizes App Store Connect asks for: 6.9″ 1320×2868 and 6.3″ 1206×2622.

Captions are drawn by macOS’s own text engine, so Arabic, Hebrew, Japanese, Korean and Chinese come out right.

Terminal
PROJECT=MyApp.xcodeproj SCHEME=MyApp BUNDLE=com.you.myapp \
  ./capture.sh en de ja ar
python3 compose.py

Also included: OnDeviceDemo, a small SwiftUI app that downloads LFM2 2.6B and turns any text into a note, showing which tier answered and how long it took; and a Swift Testing suite whose integration tests run against a real model when ONDEVICE_MODEL points at a GGUF file.

Taken out of two apps we ship

OnDevice Kit isn’t a weekend demo. It is the code underneath two of our own iOS apps at theyplay.games, pulled out once both needed the same parts.

LAITTER

Scan a paper letter and get what it’s about, what to do and by when. Reads the letter with Apple Intelligence, LFM2 2.6B or rules, all on the iPhone. The first two letters are free (Keychain), then a one-time unlock.

Hausgeist

Record by voice how things in your home work and get clear steps for family and guests. Same three tiers, the model served from our own server with Hugging Face as fallback, App Store screenshots in ten languages.

One thing both apps taught us: a 2–3B model gets far better with one worked example in the prompt than with longer instructions. That’s why Assistant takes examples.

Models that work well

Any GGUF model with a chat template runs. These are the ones we tested; LFM2 2.6B ships as a ready ModelSpec with size and SHA-256.

ModelSize (Q4_K_M)Good atLicense
LFM2 2.6B Liquid AI1.6 GBextraction, 8 languages, fastLFM Open License (free below $10M yearly revenue)
Qwen3 4B Instruct2.5 GBreasoning, many languagesApache 2.0
Gemma 3 1B0.8 GBshort tasks on older phonesGemma Terms of Use
Llama 3.2 3B2.0 GBEnglish textLlama 3.2 Community License
~6.5 s
for LFM2 2.6B to fill a six-field letter schema on an M-series Mac
20–30 s
first load of the 1.6 GB model; cached for every call after that
1.6 GB
LFM2 2.6B in 4-bit, downloaded once in the background and checked against its SHA-256

Check each model’s license before you ship it: your app is the one distributing the model.

Free guides

What we learned building both apps, written up with code you can use without buying anything.

Pricing

Pay once. Use it in as many apps as you like, including apps you build for clients.

Solo

$79 one-time

For 1 developer.

  • Full source of all four modules
  • Example app, tests and screenshot tools
  • Unlimited apps, free or paid
  • 12 months of updates
Buy Solo

Team

$199 one-time

For up to 10 developers in one company.

  • Everything in Solo
  • One license for the whole team
  • Unlimited apps, including client work
  • 12 months of updates
Buy Team

Checkout and invoices by Polar, prices in USD plus VAT where it applies. After the 12 months you keep every version you received. The license lets you use and modify the kit in your own apps; it does not let you resell or publish its source. llama.cpp is MIT; models carry their own licenses.

Questions

What exactly do I get?

The full Swift source of OnDevice Kit (the OnDeviceLLM, OnDeviceModels and OnDeviceStore modules plus the umbrella OnDeviceKit), the OnDeviceDemo example app, the test suite and the screenshot tools, as a Swift package you add to Xcode as a local package. llama.cpp comes in through Swift Package Manager as an unchanged build of release b11887 that adds the iOS Simulator, pinned to a version and checksum.

What does the license allow?

Use and modify the kit in any number of your own apps, free or paid, including apps you build for clients. Solo covers one developer, Team up to ten developers in one company. You may not resell, sublicense or publish the source, on its own or as part of a template, starter kit or course. Compiled apps are fine. Updates are included for 12 months, and you keep every version you received.

Does it run in the iOS Simulator?

Yes. The official llama.cpp release has no iOS Simulator slice, so the kit links its own unchanged build of the same release that includes one. In the Simulator the local model runs on the CPU, which takes a minute or two for a 2–3B model instead of seconds: good enough to build your UI, use a device for real speed. Apple Intelligence in the Simulator needs it turned on on your Mac.

Which iPhones can run a local model?

A 4-bit 2–3B model such as LFM2 2.6B (1.6 GB) needs an iPhone with 6 GB of RAM or more: iPhone 13 Pro and Pro Max, iPhone 14 and newer. For older phones, a 1B model like Gemma 3 1B (0.8 GB) handles short tasks, and the rules tier works everywhere. Apple Intelligence needs an iPhone 15 Pro or newer on iOS 26 with the feature turned on; AppleModel.unavailableReason tells you why it isn’t available.

Will App Review accept an app that downloads a model?

What we do for our own apps: say in the review notes that the app downloads a language model on request and runs it on the device, keep the download optional or explain it during onboarding, show its size before downloading, and link the model license in settings. The docs have the details.

Does any data leave the device?

No. The kit sends nothing anywhere and contains no analytics. Its only network request is the model download from the URLs you configure; purchases go through Apple’s StoreKit as usual.

Who is responsible for model licenses?

You are, because your app distributes the model. Models aren’t part of the kit. LFM2 uses the LFM Open License (free commercial use below $10M yearly revenue), Qwen3 is Apache 2.0, Gemma and Llama have their own terms. Read the license of the model you pick before you ship.

What do I need to build with it?

Xcode 26 and Swift 6. Apps can target iOS 17 and macOS 14 or later; Apple Intelligence is used automatically on iOS 26 and macOS 26 where it’s available. You can take everything through OnDeviceKit or only the modules you need.

Can I get a refund?

Because you get the full source code right away, purchases are final. If the kit doesn’t work as described on your setup, email info@theyplay.games within 14 days. We’ll help you fix it, or refund you if we can’t.

Ship the AI feature, not the plumbing.

Schema, three tiers, model download, free quota, unlock and screenshots, in one package you own.