How a Model Is Made: Training, Fine-Tuning, LoRA and Friends

How does a model get its skills? Not in one step. A model goes through stages, like a person going from school to a job. This lesson walks through the stages in plain words. Then it explains three tricks that make models smaller or cheaper: LoRA, distillation and quantization.

Stage one: pretraining

First the model reads an enormous amount of text. Its only task is to guess the next token. By doing this billions of times, it learns grammar, facts, styles and patterns. The result is a base model. It knows a lot, but it is not helpful yet. Ask it a question, and it may just continue the text.

Stage two: instruction tuning

Next, people show the model many examples of a question and a good answer. The model learns to follow instructions. Now it behaves like an assistant.

Stage three: feedback

Then people, and sometimes other AI systems, compare answers. They mark which answer is better. The model learns to prefer helpful, honest and safe replies. You will often see this called learning from human feedback. This is why chat models are polite and refuse some requests.

Extra stage: fine-tuning

Fine-tuning means more training on your own examples. A company might train on thousands of its own support replies. The model then copies that style and format better. Fine-tuning is good for steady behaviour. It is not a reliable way to add many new facts.

Three tricks for smaller and cheaper

• LoRA: instead of changing the whole model, you train a small add-on. It is cheaper and quicker, and you can swap add-ons.
• Distillation: a big teacher model helps train a small student model to copy its answers. The student is faster and cheaper.
• Quantization: store the model's numbers with fewer digits. The model gets smaller and runs faster. A little quality may be lost.

Why you should care

• It explains why models know a lot, yet may be out of date. Training stops at some point.
• It explains why chat models are polite. Feedback shaped them.
• It helps you choose between a big model and a tuned small one for a job.

A simple example

Say a clinic wants an assistant that writes patient letters in a calm, clear style. It starts with a general chat model. Prompts get close but not close enough. So the clinic fine-tunes on a few thousand good letters. Now the style is steady. To keep it cheap, the clinic uses a small tuned model rather than the biggest one. Each choice came from one stage on this page.