AI & LLM Foundations for Engineers

This course follows a textbook on AI engineering. It takes you from I can call an AI service to being able to design, build, secure, test, launch and run AI systems. You will also learn to explain the trade-offs in a senior interview. Tools and code details will change. The principles in these lessons are meant to last.

The field as a stack of layers

The most useful picture is that an AI application is a stack of layers. The model writes and reasons. Context gives it what it does not reliably know. Search finds the right information. Tools let it act on other software. Workflows control how several steps run. Agents add choices made on the fly. Production engineering, which means permissions, testing, monitoring, reliability and cost, makes the whole thing trustworthy.

• Foundation model: what can it write or reason about? Think about the choice of model, speed, context size and cost.
• Prompt and context: what information and instructions reach it? Think about templates, fixed formats and limits.
• Search (retrieval): where does up-to-date or specialist knowledge come from? Think about chunking, embeddings, ranking and sources.
• Tools: what can the system actually do? Think about schemas, logins, safe repeats and permissions.
• Workflow: which sequence should always run the same way? Think about routing, chaining, parallel steps and retries.
• Agent: which decisions can be handed over while it runs? Think about plans, loops, saved state and when to stop.
• Protocol: how do abilities connect in a standard way? Think about MCP, API contracts and identity.
• Production: can it be trusted at scale? Think about tests, security, monitoring, service goals and cost.

AI, machine learning and deep learning

Artificial intelligence is the wide goal: systems that do things we link with intelligence, like seeing, predicting, reasoning, planning, understanding language and acting. Machine learning gets some of that by learning patterns from data instead of following hand-written rules. Deep learning is machine learning built mostly from neural networks with many layers. You do not need to become a researcher. You need enough theory to predict how a system behaves, to know its limits, to choose a technique and to explain the trade-offs.

Know the difference between training and inference. Training changes the model's settings, called parameters, using data. Inference uses the trained model to answer a new input. Most application engineers spend their time on the inference side: prompts, search, tools, saved state, testing, security and serving.

Choose the kind of solution before the technology

When a company says we need AI, the first question is what kind of problem it is. The wrong choice leads to needless complexity.

Prediction problem   "Will this customer churn?"            -> classical ML or a simple model; an LLM is rarely needed
Generation problem   "Draft a reply to this ticket"          -> LLM call with good context and a review step
Retrieval problem    "What does our policy say about X?"     -> RAG / search; grounding matters more than generation
Workflow problem     "Extract, validate, then file a claim"  -> deterministic pipeline with LLM steps where language is involved
Action problem       "Issue the refund if eligible"          -> tools + authorization + (often) human approval

Ask first: which of these is it, and what is the SMALLEST technique that solves it?

Neural networks and transformers, in everyday terms

A neural network is a function made of layers, with many adjustable settings. In training, the model compares its answer with the right one and measures how wrong it was. Then it nudges its settings to be less wrong. Transformers changed language models because of attention. With attention, each token can look at the other tokens in the text and decide which ones matter for it. Several attention heads each look for a different kind of link. Other layers reshape the information. Some tricks keep very deep networks stable.

Why this matters to you: it explains why context length, tokenization and the cost of attention matter. It also explains why giving a model more text is not the same as giving it better information. A long context can hold noise, clashing instructions and irrelevant passages. Newer models also handle images and audio. So design your inputs and outputs as structured data, and do not assume text only.

Tokens, context windows and embeddings

• A token is a unit the model works with, such as part of a word or a punctuation mark. Costs and limits are counted in tokens, not characters. Code and languages other than English usually need more tokens.
• The context window is how much input plus output the model can handle in one request. It is not memory. The model remembers nothing after the request, unless your app saves the state and sends it again.
• An embedding turns content into a list of numbers, so that similar meanings sit close together. Searching by similarity, often with cosine similarity, powers most RAG systems. The model that makes embeddings is a different model from the one that writes text. Do not mix them up.
• In production, keep labels next to the vectors: customer, document, access rules and freshness.

import math

def cosine(a: list[float], b: list[float]) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))

# 1.0 = same direction (very similar), 0 = unrelated, negative = opposite.
# Quality depends on the embedding model, how you chunk, and metadata filters - not just the vector database.