All posts

I try to create a system that helps me with my ongoing struggle of outsourcing all my thinking to LLMs. (WiP)

A technical walkthrough of the transformer attention block, the parallelism strategies used to distribute it (DP, TP, CP, PP), mixture-of-experts dispatch and combine, and the collective communication primitives and cost models underneath all of it.

A reliable-UDP proxy and Chandy-Lamport based snapshotting system for checkpointing and restoring live, distributed containerized applications.

Maybe it is just me, but starting with a filled table and working backwards has never been how my brain wants to approach these problems.

Most FL strategies that claim to improve on FedAvg do not actually improve on FedAvg. Not on real hardware. Not under real conditions.

Benchmarking RMSNorm and Flash Attention across Helion, Triton, Gluon, and CuTe on four NVIDIA GPU generations.

A performance characterization of the Rockchip RK3588S SoC, exploring the gap between theoretical hardware limits and observed system behavior.

The Rudrashtakam is a Sanskrit hymn composed by Tulsidas in praise of Bhagavan Shiva, invoking His formless, compassionate, and all-pervading nature.

An overview of Google Spanner’s architecture, distributed transactions, and the TrueTime API, based on the paper by Corbett et al.

An overview of the Google File System (GFS), its architecture, design decisions, and the assumptions behind it, based on the paper by Ghemawat et al.