Cloud LLMs: Microsoft Foundry and Amazon Bedrock

On-device inference is private and free; cloud inference is more capable and scales past any laptop. Real applications use both — on-device where it suffices, cloud where it doesn’t. This final article wires up two big cloud providers, Microsoft Foundry (Azure OpenAI) and Amazon Bedrock, behind the same IChatClient, and builds the factory that routes between all three tiers — local, on-device and cloud — with the application none the wiser.

If the series has done its job, none of this will surprise you.

The series

The full source code of this series is available on GitHub.

One factory, one interface

Everything lives in ChatClientFactory.cs. Every branch returns the same IChatClient, so the code above the factory never branches on provider:

public static IChatClient Create(AiProvider provider) => provider switch
{
    AiProvider.Ollama        => CreateOllama(),
    AiProvider.AzureFoundry  => CreateAzureFoundry(),
    AiProvider.AmazonBedrock => CreateAmazonBedrock(),
    _ => throw new ArgumentOutOfRangeException(nameof(provider))
};

That switch is the only place in the entire application that knows more than one provider exists.

Microsoft Foundry (Azure OpenAI)

static IChatClient CreateAzureFoundry()
{
    var endpoint   = Required("AZURE_OPENAI_ENDPOINT");
    var deployment = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT") ?? "gpt-4o-mini";
    var apiKey     = Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY");

    AzureOpenAIClient azureClient = string.IsNullOrWhiteSpace(apiKey)
        ? new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())  // Entra ID
        : new AzureOpenAIClient(new Uri(endpoint), new ApiKeyCredential(apiKey));

    return new ChatClientBuilder(azureClient.GetChatClient(deployment).AsIChatClient())
        .UseFunctionInvocation()
        .Build();
}

Key points:

• deployment is your deployment name, not the model name. In Azure you deploy a model under a name you choose; that name is what you pass to GetChatClient.
• .AsIChatClient() adapts Azure’s ChatClient to the abstraction — the same adapter idea as .AsIImageGenerator() in article 7.
• Auth prefers DefaultAzureCredential (managed identity, az login, etc.) and only falls back to a key if you supply one. No secrets in code.
• .UseFunctionInvocation() is applied here too, so the tools from article 3 work identically against the cloud model.
• AZURE_OPENAI_ENDPOINT is not a value you invent — Azure derives it from the resource name, as https://<your-resource>.openai.azure.com/. Article 7 walks through creating the resource, deploying a model and granting your DefaultAzureCredential identity the role it needs; the steps are identical here, only the deployed model differs.

Amazon Bedrock

static IChatClient CreateAmazonBedrock()
{
    var modelId = Environment.GetEnvironmentVariable("BEDROCK_MODEL_ID")
                  ?? "anthropic.claude-3-5-sonnet-20241022-v2:0";
    var region  = Environment.GetEnvironmentVariable("AWS_REGION") ?? "us-east-1";

    // Credentials come from the default AWS chain: env vars, ~/.aws/credentials,
    // SSO, or the instance role.
    var runtime = new AmazonBedrockRuntimeClient(RegionEndpoint.GetBySystemName(region));

    return new ChatClientBuilder(runtime.AsIChatClient(modelId))
        .UseFunctionInvocation()
        .Build();
}

The bridge is AWSSDK.Extensions.Bedrock.MEAI, which adds AsIChatClient(modelId) to AmazonBedrockRuntimeClient. Behind it, GetResponseAsync / GetStreamingResponseAsync map onto Bedrock’s Converse / ConverseStream APIs. Bedrock is a marketplace — the modelId selects the vendor (Anthropic Claude here, but also Meta Llama, Mistral, Amazon Titan and others). Credentials resolve through the standard AWS chain, so there is nothing provider-specific to wire up if your environment is already configured for AWS.

The application never changes

Program.cs picks a provider from the command line and then runs a loop that is byte-for-byte independent of that choice:

using IChatClient client = ChatClientFactory.Create(provider);

// From here down, nothing knows or cares which provider is running.
await foreach (var update in client.GetStreamingResponseAsync(prompt))
    Console.Write(update.Text);

Set the variables for the provider you want first. Ollama needs none; the two cloud providers do. Bash:

export AZURE_OPENAI_ENDPOINT="https://<your-resource>.openai.azure.com/"
export AZURE_OPENAI_DEPLOYMENT="gpt-4o-mini"    # your deployment name
# optional: export AZURE_OPENAI_API_KEY="..."   (omit to use az login)

export BEDROCK_MODEL_ID="anthropic.claude-3-5-sonnet-20241022-v2:0"
export AWS_REGION="us-east-1"

PowerShell has no export — environment variables live on the env: drive:

$env:AZURE_OPENAI_ENDPOINT = "https://<your-resource>.openai.azure.com/"
$env:AZURE_OPENAI_DEPLOYMENT = "gpt-4o-mini"    # your deployment name
# optional: $env:AZURE_OPENAI_API_KEY = "..."   (omit to use az login)

$env:BEDROCK_MODEL_ID = "anthropic.claude-3-5-sonnet-20241022-v2:0"
$env:AWS_REGION = "us-east-1"

Both set the variables for the current session only; see article 7 for persisting them, and prefer dotnet user-secrets once this stops being an experiment. AWS credentials themselves stay where the SDK expects them — aws configure, SSO, or an instance role — not in these variables.

Then run the same program three ways:

dotnet run --project 09.CloudLLMs -- ollama
dotnet run --project 09.CloudLLMs -- azure     # needs AZURE_OPENAI_* env vars
dotnet run --project 09.CloudLLMs -- bedrock   # needs AWS creds + BEDROCK_MODEL_ID

Same prompts, same loop, three different backends. This is the entire thesis of Microsoft.Extensions.AI demonstrated in one file.

Production concerns

Cloud calls cross a network to a rate-limited, occasionally-flaky, metered service. Two things belong on that path.

Resilience. Microsoft.Extensions.Http.Resilience adds retry with backoff, a circuit breaker and timeouts. Transient 429/503 responses are normal at scale; retrying them is not optional. Add the standard resilience handler to the HttpClient your provider uses.

Observability. UseOpenTelemetry() on the pipeline emits traces and metrics for latency, token usage and errors — the numbers that turn into a bill and a latency SLO. The Usage metadata from article 1 flows through here.

Choosing a tier

The mature answer is hybrid: probe for an on-device model, use a local server in development, and reach for the cloud when you need frontier capability or scale. The ChatClientFactory here is the seam where that policy lives — and because it returns IChatClient, changing the policy never ripples outward.

Series wrap-up

Nine articles, one idea, applied over and over:

Program against the abstraction. The provider is a registration detail.

You built a chat client (1–2), gave the model your own tools (3), tested it deterministically (4), taught it to answer from your documents (5–6), generated images (7), moved inference onto the device (8), and routed to the cloud (9) — and through all of it the application code in front of the interface barely moved. Chat, embeddings, RAG, images, local, on-device, Azure, AWS: same IChatClient, same IEmbeddingGenerator, same IImageGenerator.

That is what Microsoft.Extensions.AI buys you — the freedom to change your mind about providers, and the models beneath them, without rewriting your app.