The AI Glossary

Large Language Model (LLM)

A model trained on enormous amounts of text to predict what word (technically, what "token") comes next in a sequence. That simple prediction task, done at massive scale, is what produces everything from chatbots to code assistants — the model isn't "looking up" answers, it's generating them one token at a time based on patterns learned during training.

Parameter (Parameter Count)

A single adjustable number inside the model, tuned during training to help it make better predictions. Modern LLMs have anywhere from a few billion to over a trillion parameters. More parameters generally means more capacity to capture nuance, but also more compute and memory required to run — it's a rough proxy for a model's size, not a direct measure of how "smart" it is.

Training vs. Inference

Training is the one-time (very expensive, often months-long) process of teaching a model by having it predict text across a huge dataset and adjusting its parameters when it's wrong. Inference is every single time after that when someone actually uses the finished model to generate a response. Training happens once per model version; inference happens billions of times a day — which is why inference cost and speed, not just training cost, are what most AI infrastructure spending is increasingly aimed at.

Fine-Tuning

Taking an already-trained general-purpose model and training it further on a smaller, focused dataset to specialize it — e.g. taking a general LLM and fine-tuning it on legal documents to make it better at legal work. Far cheaper than training a new model from scratch, since it builds on capabilities the base model already has.

Context Window

The amount of text (measured in tokens) a model can "see" at once when generating a response — both what you've given it and what it has already written in the current conversation. A larger context window lets a model work with longer documents or conversations, but everything in it has to be reprocessed on every turn, which is a major driver of inference cost at scale.

Retrieval-Augmented Generation (RAG)

A technique where, before generating an answer, the system first searches an external database or document set for relevant information and feeds it into the model's context window. This lets a model answer questions about specific, current, or private information it was never trained on, and is one of the main ways companies build AI tools on top of their own data without retraining a model.

Hallucination

When a model confidently generates information that sounds plausible but is factually wrong or entirely made up — a fabricated citation, a nonexistent API function, a wrong date. It happens because the model is fundamentally predicting plausible-sounding text, not looking up verified facts, which is exactly the gap RAG is often used to narrow.

Inference Speed / Latency

How quickly a model produces its response once it starts generating. Usually measured in tokens per second, or as "time to first token." This is a genuine hardware and engineering problem — it's the reason companies design specialized chips and serving software, since a slow, laggy AI product is a much harder sell regardless of how capable the underlying model is.

GPU (Graphics Processing Unit)

A chip originally built to render video game graphics, which turns out to be extremely good at the kind of massively parallel matrix math that AI models rely on for both training and inference. Nvidia's dominance of the AI hardware market is built primarily on GPUs and the CUDA software layer that runs on them — see Lesson 1 of AI Infrastructure 101 for more.

ASIC (Application-Specific Integrated Circuit)

A chip custom-designed to do one specific job extremely efficiently, instead of being general-purpose like a GPU. Google's TPUs are the best-known AI example — by giving up a GPU's flexibility, an ASIC can often run a specific model architecture faster and at lower power cost. See Lesson 2 of AI Infrastructure 101.

TPU (Tensor Processing Unit)

Google's custom AI ASIC, purpose-built for the "tensor" math operations that underlie neural networks. Used both internally (to run Google's own models like Gemini) and rented out via Google Cloud as an alternative to Nvidia GPUs.

HBM (High-Bandwidth Memory)

A type of memory stacked physically close to a chip so data can move between them extremely fast — critical for AI accelerators, which are frequently bottlenecked not by raw compute power but by how quickly they can feed data into it. HBM supply has been a recurring semiconductor-industry chokepoint, since only a handful of manufacturers (SK Hynix, Samsung, Micron) can produce it at the volumes AI hardware demands.

Data Center / Hyperscaler

A data center is a facility purpose-built to house racks of servers, with the power and cooling infrastructure to run them. A "hyperscaler" is one of the handful of companies (Microsoft, Amazon, Google, Meta) that operate data centers at a scale large enough to rent out computing capacity to everyone else — the underlying physical infrastructure that essentially all cloud AI services run on top of.

TAM (Total Addressable Market)

The total revenue a company could theoretically capture if it won 100% of the market it's selling into. Useful for sizing how big a growth opportunity genuinely is, but treat any TAM figure skeptically — it's one of the easiest numbers in a pitch deck to inflate by defining the market too broadly.

Moat

Whatever makes it hard for a competitor to take a company's customers even after building something just as good — switching costs, network effects, and scale advantages are the three most durable kinds. Covered in depth in Valuing Tech Moats, Lesson 3.

Rule of 40

A SaaS-industry shorthand: add a company's revenue growth rate to its profit margin. If the total clears 40%, the business is considered healthy, whether that 40 comes mostly from growth or mostly from margin. Covered in Valuing Tech Moats, Lesson 2.

PEG Ratio

A company's P/E ratio divided by its earnings growth rate, used to judge whether a stock's price is reasonable relative to how fast it's actually growing — a high P/E paired with high growth can produce a lower, more reasonable-looking PEG than a low P/E paired with low growth. Covered in Valuing Tech Moats, Lesson 2.

EV/Sales (Enterprise Value to Sales)

A company's total value (market cap plus debt, minus cash) divided by its annual revenue. Useful for comparing companies that aren't profitable yet, since it sidesteps earnings entirely and just asks what the market is paying for each dollar of revenue.

ARR (Annual Recurring Revenue)

The predictable, subscription-based revenue a company can count on collecting over the next 12 months, annualized from its current run rate. The standard growth metric for SaaS and other subscription businesses, since it strips out one-time or lumpy revenue that doesn't repeat.

Gross Margin

Revenue minus the direct cost of delivering a product or service, as a percentage of revenue. For an AI company, this is heavily influenced by the compute cost of running inference — a company whose gross margin is shrinking even as revenue grows is often quietly subsidizing usage or facing rising infrastructure costs. Covered in Valuing Tech Moats, Lesson 4.

Capex (Capital Expenditure)

Money spent on long-term physical assets — for AI infrastructure companies, this overwhelmingly means data centers, chips, and power agreements. Heavy capex is exactly why hypergrowth AI companies often show weak GAAP earnings despite a genuinely strong underlying business; see Valuing Tech Moats, Lesson 1.