writing

Notes & writeups

Will put any Essays I plan to start writing, Technical/Research Blogs, and any reference Notes. These will be getting edited in place rather than being finished so there will be mistakes. Written in Obsidian, published by a git push.

Essays

Longer-form thinking, and the pieces I actually finished.

· 1 min

Rebuilding this site (and how writing works here now)

This site is a pile of Markdown files now. The old version was a React app, and adding anything meant editing components. By the time I finished fiddling with JSX, I didn’t feel like writing anymor…

site

· 3 min

Move 37, Move 78

My new revamped website’s homepage animation replays the opening of Game 4 of Lee Sedol vs. AlphaGo and stops at Move 78 which is arguably one of the most famous moves played by a human, a wedge th…

gorl

Notes

8 of them, filed by how far down the stack they sit.

Foundations

The math I forget.

note · · 6 min

Probability, likelihood, and measure

Why probability and likelihood are different objects, what MLE is actually doing, and the measure-theory vocabulary that makes densities, PMFs, and Dirac deltas one thing.

math

in progress · · 1 min

Convex optimization

Convex sets, duality, and the conditions that make an optimization problem tractable. Working through Boyd & Vandenberghe.

math

in progress · · 1 min

Information theory

Entropy, mutual information, and the coding view of the losses we already train with. Placeholder for notes in progress.

math

Models & training

What each family of model is actually optimizing, and how the pieces get put together.

note · · 3 min

What each model is actually optimizing

A map of ML paradigms by objective rather than by architecture — which ones are maximum likelihood in disguise, which are bounds, and which are control problems wearing an inference costume.

mathllm

note · · 3 min

Diffusion, GANs, and what each buys you

Why diffusion optimizes a proxy instead of a likelihood, the things GANs still do better, and the customization techniques stacked on top of image models.

diffusion

note · · 2 min

Post-training — SFT, RL, and GRPO

Why SFT is forward KL and RL is reverse KL, what actually changes between PPO and GRPO, and how the adapter methods divide up the problem.

llmrl

note · · 1 min

Distributed training and torch.compile

What DDP and FSDP actually shard, and why torch.compile is a memory and graph optimization rather than anything to do with attention.

systems

Papers & research notes

Reading-group writeups.

note · · 9 min

RL reading group

Running notes from reading group — depth scaling in contrastive RL, TD-JEPA, curriculum learning for RLVR, and skill-driven state abstractions. Discussion included, objections included.

rlpapers