Open, Closed, Small and Local Models

Layer 6 is about running AI for real. The first choice is which model, and where it runs. You can rent a model from a company through the internet. Or you can run one yourself. Each way has trade-offs. This lesson gives you a simple way to choose.

Closed models: rent from a provider

• You send text over the internet. The provider runs the model.
• Often the strongest and easiest to start with.
• You pay for use. Costs grow with traffic.
• Your data goes to the provider, so check privacy terms.
• The provider can change or retire a model.

Open-weight models: run it yourself

Some companies publish the model's weights, the numbers it learned. You can download and run them. People often call these open models. The licence still matters. Some allow business use. Some do not. Read it.

• You control the data. It can stay on your own machines.
• You can tune the model for your job.
• You need computers and people to run it.
• Quality may trail the best closed models, though the gap changes.

Small language models

A small model has far fewer parameters. It is faster and cheaper. It is often good enough for narrow jobs, such as sorting messages or extracting fields. With tricks from lesson three, such as quantization, a small model can run on a laptop or phone.

Running locally

Local means on your own device. There are tools that make this easy for a home computer. Local gives privacy and works offline. The trade-off is speed and quality, which depend on your hardware.

Questions to ask

• How sensitive is the data? Can it leave our systems?
• How hard is the task? Does it need the strongest model?
• How much traffic? Rented costs rise with use. Your own costs are mostly fixed.
• Who will run and fix it?
• What happens if the provider changes the price or the model?

It can be a mix

Many products use several models. A small model handles easy requests. A strong model handles hard ones. This is called routing. It keeps quality high and cost low.

Easy question (reset password)  -> small, cheap model
Hard question (contract advice) -> strong model, plus a human check
Private records                 -> model that runs on our own servers

A small example

A hospital wants an assistant to summarize notes. The notes are private. Sending them to an outside provider may break its rules. So the hospital runs an open-weight model on its own servers. A small clinic with public information might choose the opposite. It rents a closed model, because the data is not sensitive and the setup is quick.