Tag: ai
All the articles with the tag "ai".
Optimizers: SGD to Adam to what's next
Published: at 09:20 PMHow a network takes a step downhill, from gradient descent and momentum to Adam, AdamW, Muon and Shampoo, and which ones big labs really use.
Backpropagation as blame assignment
Published: at 09:10 PMBackpropagation is the chain rule used as a blame ledger. I work one tiny network by hand, then scale it to the 784-16-16-10 digit network.
Can a model do math it doesn't understand?
Published: at 02:50 AMSearle's Chinese Room restated for machine-written proofs, what interpretability says models actually do when they add, and why I think 'can it tell when it's wrong' is a better question than 'does it understand'.
What a neural network actually is, built up from one neuron
Published: at 08:45 PMStarting from a single neuron and a handwritten digit, I build up what a neural network is, why it needs nonlinearity, and what training does.