ML Foundations

I'm a backend engineer relearning machine learning from the ground up.

I forgot most of what I studied a few years ago, so I'm starting over and writing everything down as I go.

Following Andrew Ng's Machine Learning Specialization first, then moving to applied LLM and AI engineering work.

Notes are for me. If they help you too, good.

Progress

• Week 1 — Supervised vs unsupervised, cost function, gradient descent
• Week 2 — Regression with multiple inputs

Running Week 1

# create and activate a virtual environment (macOS Homebrew Python requires this)
python3 -m venv .venv
source .venv/bin/activate
pip install numpy matplotlib

# run from repo root
python3 Week-1-Supervised-Learning/gradient-descent.py

Charts are saved to charts/. Full written notes live in Week-1-Supervised-Learning/notes.md.

---

Week 1 Notes — Machine Learning Foundations

What is machine learning?

Teaching a computer to learn patterns from data instead of hand-coding every rule. Week 1 covers supervised learning: data includes correct answers (labels), and the model learns to map inputs → outputs.

1. Supervised vs Unsupervised

Type	Labels?	Goal	Example
Supervised	Yes	Learn input → output mapping	House size → price
Unsupervised	No	Find hidden structure	Cluster similar customers

Chart 1_supervised_vs_unsupervised.png: left = colored by known class; right = same points unlabeled — algorithm must discover groups.

2. Regression vs Classification

Both are supervised. The output type differs:

• Regression — predict a number (continuous). Example: house price from size.
• Classification — predict a category (discrete). Example: tumor benign (0) or malignant (1).

Chart 2_regression_vs_classification.png: left = price regression line; right = tumor class (0/1) vs size.

3. The Model: f(x) = w·x + b

Linear regression with one feature is a straight line:

f(x) = w * x + b

• w — slope (weight): how much output changes per unit input
• b — intercept (bias): value when x = 0
• ŷ or f(x) — the prediction

Dataset used in code:

Size (1000 sqft)	Price ($1000s)
1.0	300
1.5	340
2.0	480
2.5	520
3.0	610

Chart 3_model_lines.png: three different (w, b) guesses on the same data — parameters directly change the line.

4. Cost Function J(w, b)

One number measuring total error. Lower = better.

J(w, b) = (1 / 2m) * Σ (f(xᵢ) - yᵢ)²

For each point: predict → subtract actual → square error → average. Squaring keeps errors positive and penalizes large mistakes.

Chart 4_cost_curve.png: cost vs w (with b fixed at 100) — bowl-shaped curve; minimum = best w.

5. Gradient Descent

Algorithm that automatically finds good w and b:

1. Start with a guess (e.g. w=0, b=0)
2. Compute gradients (which direction is downhill):

∂J/∂w = (1/m) * Σ (f(xᵢ) - yᵢ) * xᵢ
∂J/∂b = (1/m) * Σ (f(xᵢ) - yᵢ)

3. Update both parameters simultaneously:

w := w - α * ∂J/∂w
b := b - α * ∂J/∂b

4. Repeat until cost stops dropping.

Result from our dataset (α=0.05, 2000 steps): f(x) = 160·x + 130

Chart 5_gradient_descent.png: left = cost dropping over steps (learning); right = final line on data.

6. Learning Rate α

Controls step size — the most important practical hyperparameter in basic gradient descent.

α	Effect
Too small (0.001)	Correct but very slow
Good (0.05)	Fast convergence
Too large (0.35)	Overshoots; cost diverges

Chart 6_learning_rate.png: three rates compared on log scale — green crawls, blue converges, red explodes.

Key vocabulary

Term	Meaning
Feature x	Input
Target y	Correct output
Parameters w, b	What the model learns
Cost J	Total error
Learning rate α	Step size per update
m	Number of training examples

Week 1 in one sentence

Find the line f(x) = w·x + b that minimizes squared prediction error, using gradient descent to walk downhill on the cost curve — and pick a learning rate that is neither too slow nor too unstable.

Next: Week 2 — multiple input features (multivariate linear regression).