Free sample lesson ยท Fine-Tuning Small Language Models
What's an SLM?
Can a model small enough to run on your laptop beat a giant AI at your specific task?
Lesson 1 of the Wondering course Fine-Tuning Small Language Models
A Model Small Enough to Carry
A small language model (SLM) is a language model with relatively few parameters.
Parameters are the internal numbers a model learns during training. An SLM typically has from a few hundred million to a few billion of them.
The largest general-purpose models have hundreds of billions.
Notice how the small model fits on one everyday machine while the giant model needs rows of servers, which is why size drives cost and where a model can run.
That size gap shapes everything else: what an SLM costs, where it can run, and which jobs it handles well.
Why Small Is Useful
Because SLMs are compact, they run on everyday hardware: a laptop, a phone, or a single modest GPU.
That brings four practical wins: lower cost, faster responses, offline use, and better data privacy.
- Runs on
- A laptop
- A phone
- A single modest GPU
- Benefits
- Lower cost
- Faster responses
- Offline use
- Better data privacy: Information does not have to leave your own machine.
Privacy improves because your information does not have to leave your own machine.
No customer text needs to travel to someone else's server.
The Trade-Off: Breadth
Small size has a price.
An SLM holds less general knowledge than a giant model. It is also weaker at open-ended, multi-step reasoning.
- Small model
- Large model
Ask a small model about any topic at all, or to work through a long chain of logic, and a large model will usually do better.
So the weakness is about breadth. Smaller does not mean useless everywhere.
Specialists Win Narrow Jobs
SLMs shine when the job is narrow and well defined: classifying text, extracting fields, or answering questions in one domain.
The key move is fine-tuning: training the SLM further on examples of that one specific task.
After fine-tuning, an SLM can match or even beat a much larger general model on that task, while costing far less to run.
- Pick narrow task: Choose one well-defined job, like sorting emails into four categories.
- Gather examples: Collect labeled examples of that specific task.
- Fine-tune: Train the SLM further on those examples.
- Run cheaply: Use it to match or beat a larger model on that task at far lower cost.
Imagine a small online shop that wants to sort customer emails into four categories: refund, shipping, product question, or complaint.
The Shop's Hypothetical Test
Suppose the shop must choose: pay per request for a giant cloud AI, or fine-tune a roughly 1-billion-parameter SLM on its own laptop.
It labels a hypothetical 2,000 past emails and fine-tunes the SLM. The task is narrow, has four fixed labels, and needs no world knowledge.
In this hypothetical test, the fine-tuned SLM sorts 94% of new emails correctly. The giant model, given instructions but no examples, gets 89%.
- Fine-tuned SLM: Trained on 2,000 labeled shop emails.
- Giant model: Confused the shop's own category names.
The giant model confuses the shop's own category names. The SLM also runs locally at no per-request cost and keeps data private, so the shop picks it.
Picking the Right Tool
Choose an SLM when the task is focused, data is available, and cost, speed, or privacy matter.
Choose a large model when you need wide knowledge or complex reasoning across many topics.
- New project
- Task focused?
- Data available?
- Cost, speed, privacy matter?
- Use an SLM: Fine-tune it on examples of your task.
- Use a large model: Best for wide knowledge or complex reasoning.
- Yes
- No
- Yes
- No
- Yes
- No
Try it: a chatbot that must answer anything from history to coding needs breadth. A tool that tags support tickets by type suits a fine-tuned SLM.
Check yourself
A small language model typically has _____ parameters.
- a few hundred million to a few billion
- several hundred billion or more
Answer: a few hundred million to a few billion
SLMs usually range from a few hundred million to a few billion parameters. The largest general-purpose models have hundreds of billions.
Place the missing question and recommendation back into the flowchart.
Answer: Cost, speed, privacy matter?, Use an SLM
When a project has a narrow focus and training data, choosing an SLM depends on whether local advantages like cost, speed, or privacy matter.
Pick an _____ when the task is narrow, examples are available, and cost or privacy matter. Pick a _____ when the job needs wide knowledge or complex reasoning across many topics.
Answer: Pick an SLM when the task is narrow, examples are available, and cost or privacy matter. Pick a large model when the job needs wide knowledge or complex reasoning across many topics.
SLMs are specialists: cheap, fast, and private on focused tasks. Large models are generalists, better when breadth or multi-step reasoning is needed.
Keep going
This is lesson 1 of 15 in Fine-Tuning Small Language Models. Next up:
- Small Language Models in Action
- Why Fine-Tune
- Fine-Tuning Alternatives
- Dataset Design
- Data Formatting
Wondering turns any topic into short lessons you can read, watch, or listen to, with practice questions and an AI tutor.