How answer engines work

Maybe you have seen this before. An engine quotes your page as a source and then recommends three competitors, but not you. Or it names you on Monday and leaves you out on Tuesday. Both come from how the engine builds an answer.

An answer engine pairs two systems.

1. A language model will generally aim to start from its training model, what it already knows. Or, it can take step 2.
2. A search step called retrieval-augmented generation, or RAG, hands the model fresh pages to study. Each one treats your content differently, which decides what you can influence.

Step 1. The model reads in tokens. A model breaks all text into tokens, small pieces of words. OpenAI's rule of thumb for English is about four characters per token, or three quarters of a word (OpenAI). Your brand name, your category and your product claims all reach the model as strings of these pieces.

Step 2. Training builds a memory. During training the model reads a huge sample of the web and adjusts billions of internal settings, called weights, so it gets better at predicting the next token. What it keeps is a compressed sense of which words, brands and ideas tend to appear together, rather than a copy of your pages or links to them. Training stops at a cutoff date, and the memory stays fixed until the lab ships a new model.

Step 3. Every answer has a size limit. The model writes each answer inside a context window, a space with a fixed token limit. The buyer's question, the chat so far and the pages the engine retrieved all have to fit inside it. Engines retrieve far more pages than will fit, so they keep the passages that look most relevant and drop the rest. Your page can be quoted or cited only when one of its passages makes that cut. What the model learned in training lives inside the model itself and takes up none of this space.

Step 4. RAG fills the window. The technique comes from a 2020 research paper by Patrick Lewis and colleagues at Facebook AI, which paired a model's built-in memory with a searchable index of documents (Lewis et al., NeurIPS 2020). Answer engines run a version of it on the live web.

1. Search. The engine decides whether a prompt needs fresh information and, if so, runs one or more searches. Google's developer documentation says the model makes that call itself. Chapter 3 shows these searches up close.
2. Chunk. Retrieved pages get split into shorter passages, so the engine can work with the useful part of a long page.
3. Match by meaning. Each passage gets converted into an embedding, a list of numbers that captures what the passage means. The engine compares those numbers with the prompt's own embedding and keeps the passages closest in meaning. Many systems then rerank the shortlist with a second model.
4. Write and cite. The surviving passages go into the context window. The model writes its answer from them and links the sources it drew on as citations.

What this means for your pages. Engines judge your site one passage at a time. A passage that names your product, its category and who it suits in plain words matches more prompts and still makes sense when lifted out of the page. Facts hidden in tabs, images or scripts can miss the window entirely, which chapter 4 covers. This guide uses the word retrieval for this whole live search.

A brand can drop out at any of retrieval's four steps.

ChatGPT often runs a category search first and then searches each vendor by name to check the details. Rankings carry a lot of weight in what it picks. In The Search Spot study of 488 product pages, pages ranking in Google's top 10 for five or more searches were cited regularly 76% of the time, against 10% for pages below that mark. Brands that rank in the top 10 for a wider set of relevant keywords are far more likely to be cited.

Chapter 3 goes deep on the splitting step (query fan-outs) and how to see the searches an engine runs for your category.

Important definitions for you to understand

We treat recommendations as the main goal and citations as the path to them. Chapter 8, on measurement, shows how to track each one.

Both memory and retrieval shape an answer. We focus on retrieval, because it responds to work within weeks (or days, sometimes!).

What the web says about you today is what a future model learns from, so retrieval work done now becomes memory later.

Rankings carry a lot of the load. AirOps found that 43.2% of pages ranking first on Google were cited by ChatGPT, 3.5 times the rate of pages beyond the top 20. Most of my studies have found the same outcome.

The engines share the same four steps and differ in where they search and how much they show. Treat each one as its own channel.

Ask the same question twice and you can get two different shortlists. Rand Fishkin of SparkToro and Patrick O'Donnell of Gumshoe had about 600 volunteers run 12 prompts close to 3,000 times. The chance of ChatGPT or Google returning the same list of brands twice came in under 1 in 100, and the same list in the same order came in near 1 in 1,000.

A trend still shows through the variation. That trend is what we measure. Three things cause the variation.

That is why we track prompts at the level of a category or theme. Buyers phrase things in endless ways, so a single prompt tells you little, while a group of prompts on one theme shows which way you are moving.

Reasoning mode moves the result further. In the Growth Memo test, only 25.6% of cited domains appeared under both minimal and high reasoning.