On-device LLMs with Microsoft.Maui.Essentials.AI

Everything so far has run inference somewhere else: Ollama on your machine, or a cloud endpoint. But modern phones and Macs ship their own neural hardware and their own on-device models. Running inference there means: it works offline, data never leaves the device, there is nothing to bill, and there is no network latency. For a mobile app that handles anything sensitive, that is a big deal.

Microsoft.Maui.Essentials.AI exposes Apple Intelligence (and, over time, other platform models) through the same IChatClient and IEmbeddingGenerator you already use. So the on-device switch touches exactly one place: registration.

Build note. This needs the .NET MAUI workload, and Apple Intelligence needs a physical iOS 26 / macOS 26 device — it does not exist in the simulator or in CI. So 08.OnDeviceLLMs is a source excerpt to paste into a MAUI project, not a standalone dotnet run app. Everything else in the series is unchanged; that is the whole point.

The series

The full source code of this series is available on GitHub.

Capability probing: the core pattern

You cannot assume the on-device model exists. Apple Intelligence requires recent OS versions and real hardware. So you probe, then fall back. From MauiProgramSnippet.cs:

builder.Services.AddChatClient(static _ =>
{
    if ((OperatingSystem.IsIOSVersionAtLeast(26) || OperatingSystem.IsMacCatalystVersionAtLeast(26))
        && DeviceInfo.Current.DeviceType == DeviceType.Physical)
    {
        return CreateAppleIntelligenceChatClient();
    }

    return CreateOllamaChatClient();   // Windows, Android, simulator, older iOS
});

The check is deliberately strict — both an OS-version gate and a physical-device gate — because the simulator advertises the OS version but does not host the model. On anything that fails the check you fall back to the local Ollama client from earlier articles.

Embeddings gate differently. Apple’s NLEmbeddingGenerator has existed since iOS 13, so its window is much wider than the chat model’s:

builder.Services.AddEmbeddingGenerator(static _ =>
{
    if ((OperatingSystem.IsIOSVersionAtLeast(13) || OperatingSystem.IsMacCatalystVersionAtLeast(13, 1))
        && DeviceInfo.Current.DeviceType == DeviceType.Physical)
    {
        return CreateAppleEmbeddingGenerator();
    }

    return CreateOllamaEmbeddingGenerator();
});

Creating the platform clients

The factories carry [SupportedOSPlatform] attributes and #if guards so platform-specific types are only referenced where they exist — this is what stops the Android build from trying to load an Apple framework:

[SupportedOSPlatform("iOS26.0")]
[SupportedOSPlatform("macos26.0")]
[SupportedOSPlatform("MacCatalyst26.0")]
static IChatClient CreateAppleIntelligenceChatClient()
{
#if IOS || MACCATALYST
    // Implements IChatClient over Apple's Foundation Models framework.
    // No endpoint, no API key, no network -- the model is part of the OS
    // and runs on the Neural Engine.
    return new AppleIntelligenceChatClient();
#else
    throw new PlatformNotSupportedException("Apple Intelligence is only available on iOS and macOS.");
#endif
}

[SupportedOSPlatform("iOS13.0")]
[SupportedOSPlatform("macos10.15")]
[SupportedOSPlatform("MacCatalyst13.1")]
static IEmbeddingGenerator<string, Embedding<float>> CreateAppleEmbeddingGenerator()
{
#if IOS || MACCATALYST
    return new NLEmbeddingGenerator();
#else
    throw new PlatformNotSupportedException("NLEmbeddingGenerator is only available on iOS and macOS.");
#endif
}

AppleIntelligenceChatClient and NLEmbeddingGenerator both come from Microsoft.Maui.Essentials.AI and both implement the standard interfaces. That is the entire integration: construct the right type, return it typed as the abstraction.

The Android emulator gotcha

The fallback endpoint is not always localhost:

static Uri GetLocalOllamaEndpoint()
{
#if ANDROID
    return new Uri("http://10.0.2.2:11434");  // host machine from inside the emulator
#else
    return new Uri("http://127.0.0.1:11434");
#endif
}

Inside the Android emulator, localhost is the emulator itself; the host machine running Ollama is reachable at 10.0.2.2. This trips up nearly everyone the first time.

Why the rest of the app doesn’t change

This is the payoff of eight articles of discipline. Because ChatClientServices, PdfIngestionService, ImageGenerationService and the view models all depend on IChatClient / IEmbeddingGenerator and never on a concrete provider, moving to on-device inference changes nothing above the registration. The same chat loop, the same RAG pipeline, the same tests — now running on the Neural Engine, offline, private.

Swapping the provider is genuinely a few lines in MauiProgram. Everything else you built survives intact.

Trade-offs to weigh

• On-device wins: offline, private, free per-call, low latency.
• On-device costs: smaller/less capable models than frontier cloud ones; availability gated by OS and hardware; capabilities (e.g. tool calling) vary by platform.
• Practical answer: hybrid. Probe for the on-device model, use it when present, and fall back to cloud or local-server inference otherwise — which is exactly what the registration above does, and exactly what article 9 generalises.

What you learned

• Microsoft.Maui.Essentials.AI surfaces Apple Intelligence as ordinary IChatClient / IEmbeddingGenerator.
• Always probe (OS version and physical device) and fall back; the on-device model is not guaranteed.
• [SupportedOSPlatform] + #if keep platform types out of the wrong builds; remember 10.0.2.2 for the Android emulator.
• The application layer is untouched — on-device is a registration decision.

Next: Cloud LLMs — the other end of the spectrum, and how to route between local, on-device and cloud from one factory.