shyam pathak field notes from the hidden layer
essays, paper notes, and small projects on machine learning — mostly transformers, training, evaluation, and the strange shapes inside their hidden representations.
-
What We Owe the Made
On substrate, authorship, and the ontological crime of building intelligences we refuse to look at.
-
Reward gating is killing your gradients — here's how to fix it
On GDPO, adaptive reward weighting, and getting denser gradient signal out of multi-reward RL without inviting reward hacking.