Skip to content

Linear regression: normal equation vs SGD#2

Merged
ThomasHartDev merged 1 commit into
mainfrom
thomas/feat/linear-regression
Jul 19, 2026
Merged

Linear regression: normal equation vs SGD#2
ThomasHartDev merged 1 commit into
mainfrom
thomas/feat/linear-regression

Conversation

@ThomasHartDev

Copy link
Copy Markdown
Owner

adds linear regression solved two ways so you can see the tradeoff side by side. one path is the normal equation, the exact closed form where you set the gradient to zero and solve a small linear system, with optional ridge and a pseudoinverse fallback when the design matrix is collinear. the other is minibatch gradient descent, which standardizes the features first so a single learning rate works and then maps the weights back to the raw scale. tests check that both recover the true coefficients on clean data and land in the same place on noisy data, plus the usual edge cases (empty input, rank-deficient X, bad hyperparams). mostly a demo of ordinary least squares and why you'd reach for the iterative version once the feature count gets big.

@ThomasHartDev
ThomasHartDev merged commit 73380fe into main Jul 19, 2026
2 checks passed
@ThomasHartDev
ThomasHartDev deleted the thomas/feat/linear-regression branch July 19, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant