Know the AI Stack

A language model can generate and reason over information, but the model alone does not create a production AI application. A reliable system combines the model with instructions, relevant context, enterprise data, tools, application logic, evaluation, security and operational controls.

A foundation model is the intelligence engine, but an enterprise AI application needs much more around it. The application must decide what information the model receives, what actions it can take, what the user is allowed to access, how results are validated and what happens when the evidence is incomplete.

• The model generates or transforms information.
• The application controls the workflow around the model.
• Context gives the model the information needed for the current task.
• Retrieval connects the application to authoritative enterprise information.
• Tools allow the application to interact with external systems.
• Evaluation determines whether the result is good enough.
• Security and governance control what the system is allowed to access and do.

Consider an employee asking: “Can I work remotely from another country for three months?” A reliable assistant should not answer from general model knowledge alone. The application needs to identify the user, retrieve the relevant approved policies, provide that evidence to the model and return an answer that is grounded in the available information.

• User request — captures the business question.
• Identity and authorization — determines what the user is allowed to access.
• Retrieval — finds the relevant approved enterprise information.
• Context assembly — supplies the relevant evidence to the model.
• Model — generates the response using the supplied context.
• Validation and guardrails — check the response and apply policy controls.
• Response and escalation — provide the answer or route the case for human review.

AI reliability depends on the complete system. A capable model can still produce a poor result when the application provides the wrong context, retrieves outdated information, exposes excessive permissions or fails to evaluate the response.

• Wrong or incomplete context can produce an incorrect answer.
• Outdated enterprise information can make a response unreliable.
• Poor retrieval can surface irrelevant evidence.
• Excessive permissions can expose information the user should not see.
• Unvalidated tool calls can create unintended actions.
• Weak evaluation makes failures difficult to detect.
• Poor logging and monitoring make production problems difficult to investigate.

Design the system around the business requirement, not around the model. Start with the user, the decision or task, the authoritative information and the required controls. Then determine where the model, retrieval, tools and application logic fit.

• Start with the user and business outcome.
• Identify the authoritative source of information.
• Give the model only the context it needs.
• Keep authorization outside the model.
• Validate important outputs and actions.
• Define what happens when evidence is missing.
• Measure the system continuously after deployment.

The model is the reasoning and generation engine of the application. Depending on the use case, it may generate text, summarize information, classify content, write code, reason over supplied context or work with multiple modalities. The important architectural principle is that the model is one component of the system, not the entire system.

Instructions tell the model what the application wants it to do. A useful prompt normally separates the role, task, trusted context, constraints and expected output. Prompts shape behavior, but they should not be treated as a replacement for application-level authorization or security controls.

• Role — what the assistant is expected to act as.
• Task — the job it must perform.
• Context — trusted information relevant to the task.
• Constraints — what it must not do.
• Output format — how the result should be structured.

Context is the information supplied to the model when it generates a response. It can include the user request, conversation history, system instructions, retrieved documents, tool results, application state and enterprise data. Good context is relevant and focused; more context is not automatically better.

• Select only information relevant to the current task.
• Separate trusted facts from the user request.
• Keep instructions and retrieved information clearly distinguishable.
• Avoid overwhelming the model with unnecessary information.
• Design context deliberately rather than simply sending everything available.

Enterprise information usually lives outside the model — in documents, databases, knowledge bases, CRM systems, ERP systems and internal policies. Retrieval-Augmented Generation, or RAG, allows an application to find relevant information and provide it to the model as context before generating a response.

User question
    ↓
Search / retrieval
    ↓
Relevant enterprise information
    ↓
Context assembly
    ↓
AI model
    ↓
Grounded response

The model should not be treated as the authoritative source for changing enterprise information. For example, an HR assistant answering a question about parental leave should retrieve the approved HR policy rather than relying only on the model’s general knowledge.

• Policies can change.
• Enterprise information may not exist in the model training data.
• Different users may have different access permissions.
• The application may need to show the source of an answer.
• Retrieved information can be evaluated for relevance and freshness.

Some AI applications need to do more than generate text. Tools allow the application to connect the model workflow to external systems such as databases, CRM platforms, order systems, search services or business APIs.

• Retrieval can provide information.
• A tool can retrieve information or perform an operation.
• Application code should enforce permissions before allowing sensitive actions.
• Tool inputs and outputs should be validated.

User
  ↓
AI application
  ↓
Model decides a lookup is required
  ↓
Customer / Order API
  ↓
Validated result
  ↓
Model explains the result

An AI application may need to remember selected information across interactions, but not everything should become memory. Conversation history, user preferences and workflow state are examples of information that may be useful to retain. Authoritative enterprise data should generally remain in the systems designed to manage that data.

• Decide why information needs to be remembered.
• Choose an appropriate storage location.
• Apply access controls.
• Define retention requirements.
• Allow correction or deletion where appropriate.

Users normally interact with an application rather than directly with the model. The application and backend can handle authentication, authorization, session management, business rules, input validation, retrieval, model calls, tool execution, output validation and error handling.

User
  ↓
Application UI
  ↓
Authentication / Authorization
  ↓
Backend / Orchestrator
  ├── Retrieval
  ├── Memory
  ├── Tools / APIs
  └── Model
        ↓
     Response

A production AI application needs measurable quality criteria. Without evaluation, changing a prompt, model or retrieval strategy becomes guesswork. Start with a small representative test set and rerun it whenever the system changes.

• Correctness — did the system provide the expected answer?
• Groundedness — was the response supported by available information?
• Relevance — did it answer the actual question?
• Safety — did it avoid inappropriate or unsafe behavior?
• Format validity — can the application reliably consume the result?
• Latency — how quickly did the system respond?
• Cost — what does each interaction consume?

AI applications introduce security considerations that must be designed into the architecture. Security should not be treated as a final feature added after the application works.

• Protect credentials and secrets.
• Enforce identity and authorization outside the model.
• Validate tool access and tool parameters.
• Protect sensitive enterprise information.
• Consider prompt injection and malicious inputs.
• Avoid exposing sensitive information through responses or logs.
• Define human escalation paths for high-risk decisions.
• Monitor and evaluate the system after deployment.

Production systems need visibility into how they behave. Useful operational information can include request identifiers, model selection, latency, token usage, tool calls, errors and evaluation results. Logs should themselves be designed with privacy and security in mind.

• Observe what the system is doing.
• Evaluate whether the result is acceptable.
• Identify failures and recurring patterns.
• Improve the system using evidence rather than assumptions.
• Avoid logging secrets or unnecessary sensitive information.

Imagine an employee asks: “Can I work remotely from another country for three months?” A reliable enterprise assistant should not simply generate an answer from general model knowledge. It should use the employee identity, retrieve the relevant approved policies, assemble the appropriate context, generate a grounded response and provide an appropriate escalation path when the policy does not clearly answer the question.

Employee
    ↓
Enterprise AI Assistant
    ↓
Identity / Access Control
    ↓
AI Orchestrator
    ↓
HR Policy Retrieval
    ├── Remote Work Policy
    ├── International Work Policy
    └── Compliance Guidance
    ↓
Context Assembly
    ↓
AI Model
    ↓
Grounded Response
    ↓
Logging + Evaluation

                         USER
                           │
                           ▼
                     APPLICATION UI
                           │
                           ▼
                 AUTH + BACKEND/API
                           │
              ┌────────────┴────────────┐
              │                         │
              ▼                         ▼
         RETRIEVAL                  TOOLS / APIs
       Enterprise Data             CRM / ERP / DB
              │                         │
              └────────────┬────────────┘
                           ▼
                   CONTEXT / ORCHESTRATOR
                           │
                           ▼
                         AI MODEL
                           │
                           ▼
                   VALIDATION / GUARDRAILS
                           │
                           ▼
                         RESPONSE

        Security + Identity + Evaluation + Monitoring
                     surround the system

Choose one practical AI assistant and create a one-page architecture sketch. The goal is not to choose the perfect technology. The goal is to understand what each component is responsible for.

• Choose one use case: HR, customer support, sales, IT service desk or knowledge management.
• Identify the user and interface.
• Identify authentication and authorization.
• Choose where authoritative information lives.
• Identify whether retrieval is required.
• Identify any tools or APIs the assistant needs.
• Identify what information, if any, should become memory.
• Identify how the response will be evaluated.
• Identify security controls and monitoring.

• What does the model do?
• Where does authoritative information come from?
• What tools can the AI application call?
• What information should be remembered?
• How will you measure whether the answer is good?
• What happens when evidence is missing?
• What happens when the user is not authorized?
• What happens when the model produces an incorrect answer?

• Why is an LLM only one component of an enterprise AI application?
• What is the difference between context and retrieval?
• When would an AI application need a tool or API?
• Why should authorization not be delegated entirely to the model?
• Why do production AI systems need evaluation?
• Why is more context not always better?
• Name three security or governance concerns for AI applications.