How it all connects.
This is how an AI product is actually wired: not just a model, but an app, guardrails, an orchestrator, a cache, a document pipeline, memory and tools. Scroll to follow one question through it. Click any box to read about that part.
- 1
Before any question: your documents get indexed
Help articles, policies, past tickets. They are split into chunks, each chunk is turned into an embedding (a list of numbers that captures its meaning), and the embeddings are stored in a vector database. This runs once, and again whenever the documents change.
EmbeddingsRAG - 2
You ask a question
In a chat window, a support tool or a code editor. The app takes your text and sends it on.
Prompt - 3
A guardrail checks it on the way in
Before the model sees anything, a check looks at the message: is it on topic, does it carry personal data, is it trying to override the instructions? If it passes, it reaches the orchestrator, the code that decides what happens next. This code is also called the harness, and in a product it is the "AI" part people never see.
GuardrailsPrompt injectionHarnessAI agent - 4
Has this been asked before?
The orchestrator turns your question into an embedding with the same model used on the documents, then looks in the semantic cache for an earlier question that means nearly the same. On a match, the saved answer goes straight back and everything below is skipped. This time there is no match.
Semantic cachingEmbeddings - 5
It looks up what you might need
With the same embedding, the orchestrator asks the vector database for the chunks whose meaning is closest to your question. Those chunks come back as plain text.
EmbeddingsRAG - 6
It builds the prompt
One message, assembled in order: the system prompt (who the assistant is, the rules), the conversation so far from memory, the retrieved chunks, and your question. All of it has to fit in the context window.
Context windowPrompt - 7
The model predicts a reply
The prompt goes to the LLM API. The model turns it into tokens and predicts the reply one token at a time. It does not look anything up here. Everything it knows is in its weights or in the prompt.
LLMTokenTemperature - 8
Sometimes it asks for a tool instead
If the task needs an action or live data, the model replies with a tool call, not an answer. The orchestrator runs it through an MCP server, which talks to the real system: a ticket queue, a database, a calendar. The result comes back as text.
Tool callingMCPAI agent - 9
The result goes back into the prompt
The tool result is appended and the model is called again. Think, act, observe, repeat until it has an answer. That loop is what people mean by an AI agent.
AI agent - 10
The answer is checked, streams back, and gets remembered
On the way out a guardrail checks the reply for things like made-up claims or leaked data. Some products check the whole reply first, others check it as it streams. Tokens arrive as they are generated, which is why replies appear word by word. The orchestrator saves the exchange to memory, saves the answer to the cache for the next person who asks, and logs the run so it can be evaluated later.
GuardrailsEvalsHallucinationSemantic caching