Ingesting data with IEmbeddingGenerator

A model trained last year cannot answer questions about your PDF. The fix is retrieval augmented generation (RAG): before you ask the model, you find the relevant passages in your own documents and paste them into the prompt. This article builds the retrieval half’s foundation — turning text into vectors and measuring similarity. The next article stores those vectors and closes the loop.

The abstraction is IEmbeddingGenerator<string, Embedding<float>>, and like IChatClient it hides the provider entirely.

What an embedding is

An embedding is a list of floating-point numbers (a vector) that captures the meaning of a piece of text. Texts with similar meaning produce vectors that point in similar directions, even when they share no words. That is the whole trick: it lets you search by meaning instead of by keyword.

using IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator =
    new OllamaApiClient(new Uri("http://127.0.0.1:11434"), "nomic-embed-text");

Embedding<float> sample = await embeddingGenerator.GenerateAsync("Hello, embeddings!");

Console.WriteLine($"Dimensions: {sample.Vector.Length}");

The dimension count is a property of the model, not your code. nomic-embed-text produces 768 numbers; OpenAI’s text-embedding-3-small produces 1536. Write your model’s number down; in the next article’s storage record must declare exactly the same value, and changing embedding models later invalidates every vector you have stored.

ollama pull nomic-embed-text

What is cosine?

“Cosine” is literally the trigonometry function. Cosine similarity is the cosine of the angle between two vectors.

Each embedding is a point in 768-dimensional space (that’s the Dimensions: 768 your run printed). To ask “how similar are these two texts?”, you draw an arrow from the origin to each point and measure the angle between the arrows:

• angle 0° → arrows point the same way → cos = 1.0 → identical meaning
• angle 90° → perpendicular → cos = 0.0 → unrelated
• angle 180° → opposite directions → cos = -1.0

So, the score is always in [-1, 1], and higher = more similar. Your output:

[0] vs [1]: 0.6986   "cat sat on the mat" vs "feline rested upon the rug"
[0] vs [2]: 0.2728   "cat sat on the mat" vs "quarterly revenue increased 12 percent"

The paraphrase sits at a ~46° angle, the unrelated sentence at ~74°. Nearly no shared words in either case — the angle is small for the first pair because the model placed those two vectors close together, which is the whole point of embeddings.

The formula TensorPrimitives.CosineSimilarity computes is:

cos(θ) = (A · B) / (|A| × |B|)

The dot product on top, divided by both lengths. That division is the reason to prefer cosine over plain dot product or Euclidean distance: it normalises away vector magnitude and leaves only direction. Embedding magnitude tends to track things you don’t care about (mostly text length) so without normalising, a long chunk would out-score a short, more relevant one just for being long. Cosine asks “does this point the same way?” rather than “is this big?”

A practical note for the next article: many vector stores index on cosine distance (1 – similarity) rather than similarity, so lower is better there and the ordering flips. Your OrderByDescending at Program.cs:82 is correct for similarity, but the equivalent query against a store will usually be ascending.

Similarity is cosine distance

Once two texts are vectors, “how similar are they?” becomes arithmetic. Cosine similarity returns ~1 for near-identical meaning and ~0 for unrelated. .NET has it built in via TensorPrimitives:

string[] phrases =
[
    "The cat sat on the mat.",
    "A feline rested upon the rug.",       // paraphrase of [0]
    "Quarterly revenue increased by 12 percent."  // unrelated
];

// Batch: one request for the whole array where the provider supports it.
GeneratedEmbeddings<Embedding<float>> batch = await embeddingGenerator.GenerateAsync(phrases);

var paraphrase = TensorPrimitives.CosineSimilarity(batch[0].Vector.Span, batch[1].Vector.Span);
var unrelated  = TensorPrimitives.CosineSimilarity(batch[0].Vector.Span, batch[2].Vector.Span);
// paraphrase -> ~0.8+   unrelated -> ~0.2-

Two phrases that share not one word (“cat/mat” vs “feline/rug”) score high because their meaning is close. A keyword search would miss that entirely.

Always batch. Passing the whole collection to GenerateAsync sends one request; a foreach of single calls sends N. On a cloud provider that is N times the latency and N times the billing overhead.

The ingestion pipeline

Real documents are too big to embed whole; you would blur every specific fact into one averaged vector. So, the pipeline is: extract → chunk → embed.

Extract

PdfTextExtractor.cs uses PdfPig, a fully-managed library that works on every target including mobile:

using var document = PdfDocument.Open(memoryStream);
return string.Join("\n", document.GetPages().Select(page => page.Text));

Chunk

Chunking has two knobs, and they decide whether your RAG works. From TextChunker.cs:

public const int ChunkSize = 1000;      // words per chunk
public const int ChunkOverlap = 200;    // words repeated between neighbours

for (var i = 0; i < words.Length; i += chunkSize - chunkOverlap)
{
    var chunk = string.Join(' ', words.Skip(i).Take(chunkSize));
    if (!string.IsNullOrWhiteSpace(chunk))
        chunks.Add(chunk);
    if (i + chunkSize >= words.Length)
        break;
}

• ChunkSize — too large and the vector averages away the specific fact you are hunting; too small and a chunk loses the context that gives it meaning. 500–1000 words is a sensible band for prose.
• ChunkOverlap — without it, a sentence straddling a boundary is unfindable in either chunk. ~20% overlap fixes that. The stride is chunkSize - chunkOverlap.

Embed and search

Embed each chunk once, embed the question at query time, rank chunks by cosine similarity:

var chunkEmbeddings   = await embeddingGenerator.GenerateAsync(chunks);
var questionEmbedding = await embeddingGenerator.GenerateAsync(question);

var ranked = chunks
    .Select((chunk, i) => new
    {
        Chunk = chunk,
        Score = TensorPrimitives.CosineSimilarity(
            questionEmbedding.Vector.Span, chunkEmbeddings[i].Vector.Span)
    })
    .OrderByDescending(x => x.Score)
    .ToList();

The top-ranked chunks are your retrieved context. The passages you will feed to the chat model in next article.

Run it

ollama pull nomic-embed-text
dotnet run --project 05.DataIngestion

Dimensions: 768

Cosine similarity:
  [0] vs [1]: 0.83  (paraphrase -> high)
  [0] vs [2]: 0.19  (unrelated  -> low)

Question: How do I stream a response token by token?
  score 0.71
  IChatClient exposes two methods: GetResponseAsync ... GetStreamingResponseAsync ...

The chunk that mentions streaming ranks first for a streaming question, even though the question and the chunk share almost no vocabulary. That is semantic search, and it is the engine under every RAG system.

What you learned

• IEmbeddingGenerator turns text into meaning-vectors, provider-agnostically.
• The vector’s dimension count is fixed by the model — record it, you need it again in article 6.
• Similarity is cosine distance via TensorPrimitives; you search by meaning, not keywords.
• Ingestion is extract → chunk (size + overlap) → embed, and you batch the embedding calls.