Skip to content

Logistic regression + cross-entropy loss#3

Merged
ThomasHartDev merged 1 commit into
mainfrom
thomas/feat/logistic-regression
Jul 20, 2026
Merged

Logistic regression + cross-entropy loss#3
ThomasHartDev merged 1 commit into
mainfrom
thomas/feat/logistic-regression

Conversation

@ThomasHartDev

Copy link
Copy Markdown
Owner

adds binary logistic regression trained by minibatch SGD, in the same shape as the linear regression module already here. the interesting bit is that the gradient of the cross-entropy w.r.t. the logits is just sigmoid(z) - y, the same residual you get from least squares, so the training loop looks almost identical. the sigmoid is sign-branched so exp never overflows and the loss uses softplus(z) - y*z via logaddexp so a confidently-wrong prediction gives a big finite number instead of inf, which is the usual place a naive log-loss falls over. also threw in a decision_boundary helper that gives you the 50 percent line for a 2-feature model so you can actually plot the thing. tests cover the numerical-stability edges, separable data hitting 100 percent, l2 shrinkage, and the usual bad-input guards anyway.

@ThomasHartDev
ThomasHartDev merged commit a9965e5 into main Jul 20, 2026
2 checks passed
@ThomasHartDev
ThomasHartDev deleted the thomas/feat/logistic-regression branch July 20, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant