What Is A Model¶

What This Is¶

Before any optimizer, split, or training loop, there is one question that quietly sits underneath every academy page:

what is a "model", actually, and what do I need it to learn

This page treats a model as a function with adjustable knobs. The knobs are called parameters. Training is the process that turns data into knob settings. Inference is the process of using the tuned function to produce an output.

Everything else — architectures, optimizers, loss functions, schedulers — is machinery for doing that one thing well.

This page is for absolute beginners. If you already know what gradient descent is, skip to Loss and Gradients Intuition.

When You Use It¶

you have heard the word "model" for months and want a stable mental picture
you can run a scikit-learn fit but cannot explain what .fit actually did
you are about to start Phase 4 of the Study Plan and want the deep-learning bridge
someone says "the model learned..." and you would like to know what that sentence means

A Model Is A Function¶

Pick any prediction task you can name:

given a photo, is there a cat?
given a sentence, is the sentiment positive?
given patient vitals, what is the disease risk?

A model is a function that maps the input to the output:

photo → model → "cat" or "no cat"
sentence → model → positive/negative
vitals → model → risk score

The shape of the function — how it combines the input into an answer — is the architecture. The specific numbers inside the function that make it produce this answer for this photo are the parameters or weights.

In the standard fixed-architecture workflow, training changes parameters rather than layer structure. Some algorithms also grow structure or change it during training—for example, decision trees, neural architecture search, and pruning.

The Simplest Possible Model¶

A linear model predicts a single number from an input:

y = w · x + b

x is the input (one or several numbers). w is a weight (or a vector of weights). b is a bias. Together, w and b are the parameters.

If you are predicting whether someone will buy a coffee based on the temperature, you could write:

prob = sigmoid(w * temperature + b)

sigmoid squeezes any number into the range (0, 1) so we can read it as a probability. At this point you already have a complete machine-learning model. The rest of the course is about making this same idea scale.

Parameters: The Knobs¶

In the linear model above, there are exactly 2 parameters: w and b. A neural network is the same idea, scaled up:

a tiny multi-layer perceptron (MLP) for the MNIST digits: ~20,000 parameters
a ResNet-50 for image classification: ~25 million parameters
a modern small LLM: 7 billion parameters
a large modern LLM: hundreds of billions

A model artifact can contain learned parameters as well as fixed constants, optimizer state, normalization buffers, and metadata. Learned parameters encode statistical structure from data; describing them only as memorized facts or human-like intuitions is misleading.

What Does It Mean For A Model To "Learn"¶

In supervised learning, a model learns parameters that reduce a target-based objective on training examples. Other regimes learn from reconstruction, likelihood, preferences, rewards, or unlabeled structure.

The recipe has three pieces:

Prediction — run the current parameters on an input, get an output
Loss — score the output under the training objective, often against a target
Update — nudge the parameters in the direction that reduces the loss

Repeat a lot. That is training.

Neural networks commonly use a gradient-based update rule. Other model families use closed-form solvers, greedy splits, coordinate updates, or instance lookup. Loss and Gradients Intuition walks through the gradient-based case.

A Model In 10 Lines¶

This is a complete machine-learning pipeline. Every piece here has a name that appears on dozens of later pages.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X, y = load_iris(return_X_y=True)                               # data
X_train, X_valid, y_train, y_valid = train_test_split(X, y, random_state=0)
model = LogisticRegression(max_iter=1000)                       # architecture
model.fit(X_train, y_train)                                     # training
preds = model.predict(X_valid)                                  # inference
accuracy = (preds == y_valid).mean()                            # evaluation

What happened:

LogisticRegression(...) created a function with adjustable knobs — the weights and biases
model.fit(...) adjusted those knobs so that predictions on X_train moved closer to y_train
model.predict(...) ran the tuned function on data it had never seen
comparing predictions to y_valid told us how well it generalized

This predict-and-evaluate shape describes the supervised example. Unsupervised, generative, and reinforcement-learning workflows use different training signals, though they still require an honest evaluation protocol.

Inputs, Outputs, And Labels¶

Three words you will hear everywhere:

features (also: inputs, X) — the numbers you feed into the model
label (also: target, y) — the correct answer you want the model to produce
prediction — the model's current guess for the answer

Supervised training needs inputs and labels. Other regimes may construct targets from inputs or learn from rewards or unlabeled data. Inference needs whatever inputs and state the deployed model contract specifies.

Supervised learning is learning from pairs (features, label). Unsupervised learning is finding structure in features without labels. Self-supervised learning makes up labels from the features themselves (predict the next word, predict the missing pixel).

Why Linear Models Are The Right Place To Start¶

they expose the common supervised pieces—parameters, objective, fitting, and prediction—with little extra complexity
their behavior is fully inspectable — you can literally print the weights
they are a strong floor for many problems, which makes the ladder "is the non-linear model actually beating the linear one?" meaningful
many deep-learning blocks can be decomposed into learned linear maps, nonlinear operations, normalization, routing, and connections, but architecture-specific inductive biases still matter

If a linear model already works, a deeper one is overkill. This habit is repeated on every later page.

What A Neural Network Adds¶

A linear model can only draw a single straight boundary between classes. A neural network draws many boundaries and combines them:

input → linear layer → nonlinearity → linear layer → nonlinearity → ... → output

The nonlinearity is what makes the network not collapse into a single linear function. Common choices are ReLU, GELU, and sigmoid. Without it, no matter how many layers you stack, you have still just built one big linear model.

That is the whole secret. A neural network is a sequence of simple functions with nonlinearities that lets it approximate more complicated input-output relationships.

The rest of deep learning is about making that approximation work on realistic data: choosing the architecture (CNN, transformer, RNN), choosing the training recipe (optimizer, schedule, regularization), and choosing the evaluation (splits, metrics, slices).

Evaluation Is Part Of The Model¶

A model that gets 99% accuracy on data it already saw is useless if it gets 60% on data it has never seen.

Because of this, a "model" in the academy usually means the combination of:

the architecture and trained parameters
the preprocessing applied to inputs
the split and evaluation rule that was used to decide the model is ready

The Honest Splits and Baselines page makes the evaluation part concrete.

What To Inspect After Your First Fit¶

how many parameters the model has
the accuracy on training data vs held-out data
whether a trivial "predict the most common class" model would do almost as well
a few actual predictions — right ones and wrong ones
what the model is doing wrong on the wrong ones

If you cannot answer all five, you have not trained a model; you have run some code.

Failure Pattern¶

The most common beginner failure is treating a model as a black box whose behavior is determined by the library. Every model has knobs, a loss, a training procedure, and an evaluation rule. If you cannot describe those four things for the model you just trained, you are not ready to defend any result it produces.

The second most common failure is thinking bigger automatically means better. With only 50 training examples, a huge model has much more room to overfit than a small baseline, so size must earn its place on held-out evidence.

Common Mistakes¶

calling every machine-learning system a "model" without distinguishing architecture, weights, and pipeline
confusing "learned" with "memorized"
treating a good training accuracy as evidence the model works
starting with a neural network on a problem a linear model would solve
skipping an honest split and letting the model look better than it is

Practice¶

Train a logistic regression on the iris dataset and print the weights. Explain what each weight means.
Count the parameters in your model. (Hint: sum(p.numel() for p in model.parameters()) for PyTorch, or read the coef_ shape for scikit-learn.)
Train a dummy classifier that always predicts the most common class. Compare its accuracy against your model.
Explain in one sentence what "learning" did to your model's parameters.
Pick one wrong prediction and explain why you think the model got it wrong.

Runnable Example¶

From the repository root, run .venv/bin/python examples/first-steps/first_model.py for the simplest possible training and inspection loop.

Longer Connection¶

Continue with Loss and Gradients Intuition for the update step inside .fit(), Probability and Linear Algebra Refresher for the math you will keep running into, and Honest Splits and Baselines for how to know whether your model has actually learned anything.