Training vs. Inference

2026-02-26

A common misconception about AI is that the 'thinking' happens when you ask it a question. In reality, the heavy lifting happens months earlier during a phase called 'training.' To use a metaphor: if running an AI prompt (inference) is like driving a car from New York to Boston, then training the AI is like building the entire interstate highway system from scratch. Training an LLM requires massive data centers, tens of thousands of GPUs running for months, and megawatts of electricity to lay down the 'roads' (neural network weights). Once the highway is built, anyone can drive on it. Inference is comparatively cheap and fast. The model simply runs your prompt through its pre-calculated weights to generate a response. This comic illustrates the stark contrast between the two phases. Training is a massive, industrial-scale infrastructure project, while inference is a lightweight, everyday commute. Understanding this difference is crucial for grasping the economics and environmental impact of AI.

A four-panel comic using a highway metaphor. Panel 1: A construction worker sweating while building a highway (Training). Panel 2: The worker continues laying asphalt for months. Panel 3: A person casually driving a car on the finished road (Inference). Panel 4: The driver remarks how quick and easy the trip is.
If inference is a road trip to Boston, training is building the entire interstate highway system.

Behind the Comic

Training is the process where an AI model learns patterns from vast amounts of data. It requires massive computational power (thousands of GPUs) and takes weeks or months to complete. This is when the model's 'brain' is actually built.

Inference is the process of using a trained AI model to make predictions or generate text based on new input (like a user's prompt). It requires significantly less compute power than training and happens in real-time.

Because your laptop is only doing 'inference'. It's just running data through the pre-calculated neural network weights. The supercomputer did the hard work of calculating those weights during the 'training' phase.