The AI map

How it all connects.

This is how an AI product is actually wired: not just a model, but an app, guardrails, an orchestrator, a cache, a document pipeline, memory and tools. Scroll to follow one question through it. Click any box to read about that part.

Your documents Chunking Embedding model Vector database You App (UI) Guardrails Orchestrator (harness) LLM API Memory + system prompt Semantic cache MCP servers Tools: tickets, DB, calendar 1. Indexing, done ahead of time 2. Every question 3. What the orchestrator reaches for
  1. 1

    Before any question: your documents get indexed

    Help articles, policies, past tickets. They are split into chunks, each chunk is turned into an embedding (a list of numbers that captures its meaning), and the embeddings are stored in a vector database. This runs once, and again whenever the documents change.

    EmbeddingsRAG
  2. 2

    You ask a question

    In a chat window, a support tool or a code editor. The app takes your text and sends it on.

    Prompt
  3. 3

    A guardrail checks it on the way in

    Before the model sees anything, a check looks at the message: is it on topic, does it carry personal data, is it trying to override the instructions? If it passes, it reaches the orchestrator, the code that decides what happens next. This code is also called the harness, and in a product it is the "AI" part people never see.

    GuardrailsPrompt injectionHarnessAI agent
  4. 4

    Has this been asked before?

    The orchestrator turns your question into an embedding with the same model used on the documents, then looks in the semantic cache for an earlier question that means nearly the same. On a match, the saved answer goes straight back and everything below is skipped. This time there is no match.

    Semantic cachingEmbeddings
  5. 5

    It looks up what you might need

    With the same embedding, the orchestrator asks the vector database for the chunks whose meaning is closest to your question. Those chunks come back as plain text.

    EmbeddingsRAG
  6. 6

    It builds the prompt

    One message, assembled in order: the system prompt (who the assistant is, the rules), the conversation so far from memory, the retrieved chunks, and your question. All of it has to fit in the context window.

    Context windowPrompt
  7. 7

    The model predicts a reply

    The prompt goes to the LLM API. The model turns it into tokens and predicts the reply one token at a time. It does not look anything up here. Everything it knows is in its weights or in the prompt.

    LLMTokenTemperature
  8. 8

    Sometimes it asks for a tool instead

    If the task needs an action or live data, the model replies with a tool call, not an answer. The orchestrator runs it through an MCP server, which talks to the real system: a ticket queue, a database, a calendar. The result comes back as text.

    Tool callingMCPAI agent
  9. 9

    The result goes back into the prompt

    The tool result is appended and the model is called again. Think, act, observe, repeat until it has an answer. That loop is what people mean by an AI agent.

    AI agent
  10. 10

    The answer is checked, streams back, and gets remembered

    On the way out a guardrail checks the reply for things like made-up claims or leaked data. Some products check the whole reply first, others check it as it streams. Tokens arrive as they are generated, which is why replies appear word by word. The orchestrator saves the exchange to memory, saves the answer to the cache for the next person who asks, and logs the run so it can be evaluated later.

    GuardrailsEvalsHallucinationSemantic caching