Posts
All the articles I've posted. 46 posts, page 5 of 12.
Why Adam won
Published: at 08:40 PMOne level deeper on Adam: per-parameter steps, bias correction, the sign-descent view, why it wins on language, and AdamW. With a short NumPy version.
Optimizers: SGD to Adam to what's next
Published: at 09:20 PMHow a network takes a step downhill, from gradient descent and momentum to Adam, AdamW, Muon and Shampoo, and which ones big labs really use.
Backpropagation as blame assignment
Published: at 09:10 PMBackpropagation is the chain rule used as a blame ledger. I work one tiny network by hand, then scale it to the 784-16-16-10 digit network.
Can a model do math it doesn't understand?
Published: at 02:50 AMSearle's Chinese Room restated for machine-written proofs, what interpretability says models actually do when they add, and why I think 'can it tell when it's wrong' is a better question than 'does it understand'.