Prince Modi

Prince Modi

Master’s Student, LLM Systems (Inference)

University of California San Diego

Hi, I’m Prince!

I’m a Master’s student in the Department of Computer Science and Engineering at UC San Diego, working on LLM systems (inference). Before that, I worked on scalable and resilient systems, including development of Flotilla, a modular federated learning framework, during my time at the Indian Institute of Science. While working on Flotilla, I became especially interested in the challenges of fault tolerance, consistency, and recovery in distributed systems. My broader interests span distributed computing, datacenter systems, and LLM systems.

Skills

Proficient
Python

Used extensively in development of Flotilla and various personal projects

PyTorch

Built CNNs, LSTMs, and transformer (ALBERT) models for Flotilla, now used for attention-operator profiling at Picasso Lab

Docker

Used heavily across jobs and personal projects

Linux

Daily use across work and personal setups (Arch BTW😝)

NeoVim

Daily driver, with custom plugins developed for specific workflows

Intermediate
gRPC

Integrated into a custom distributed framework using ProtoBuf definitions

Redis

Built a simplified version in Python to understand the internals

Triton

Implemented FlashAttention-2 and RMSNorm kernels, benchmarked against Helion, Gluon, and CuTe across four GPU generations

Podman

Built a reliable-UDP sidecar proxy that triggers privileged checkpoints for distributed container snapshotting

Familiar
Go

Explored through personal projects focused on distributed systems

Lua

Used to write custom NeoVim plugins and enhance editor behavior

HuggingFace

Used alongside PyTorch for transformer (ALBERT) model training and evaluation in Flotilla

SGLang

Used for disaggregated prefill/decode LLM inference serving research at Picasso Lab

Education

Master of Science in Computer Science

Graduate student at UCSD, focusing on Distributed Systems

GPA: 3.88/4.00

Relevant Courses:

  • Distributed Systems
  • Deep Learning Systems
  • LLM System Optimization

Projects:

  • System Measurement: Developed a suite of micro-benchmarks to evaluate the Rockchip RK3588S SoC, utilizing ARMv8 cycle counters to measure CPU scheduling and OS primitive latencies with nanosecond precision.(Link)
  • Distributed Container Snapshotting: Built a Chandy-Lamport based snapshotting system with a reliable-UDP sidecar proxy to checkpoint and restore live, distributed containerized applications.(Link)
  • Modern GPU DSLs: Benchmarked RMSNorm and Flash Attention-2 across Helion, Triton, Gluon, and CuTe on four NVIDIA GPU generations to evaluate performance/portability trade-offs.(Link)

Bachelor of Technology in Computer Engineering

GPA: 3.96/4.00

Achievements:

  • Academic scholarship for securing 2nd rank out of 60+ students

Relevant Courses:

  • Operating Systems
  • Computer Networks
  • Cloud Computing
  • Big Data Analytics

Projects:

  • BitTorrent Client: Built a peer-to-peer file-sharing client using Python’s AsyncIO and BitTorrent protocol with a custom Bencode parser
  • GIST: Developed a YouTube video summarizer using NLTK, BART model, SQLite, and Tkinter

Experience

Graduate Student Researcher, LLM Inference Systems

  • Profiling the attention operator in transformer LLM inference across prefill (prompt/KV-cache lengths) and decode (batch size/KV-cache lengths), fitting analytical models to measured compute and communication latencies
  • Building a trace-driven simulator over multi-turn LLM conversation traces to compare parallelism strategies (CP/TP/DP + EP) for a disaggregated prefill/decode serving deployment

Research Collaborator (Remote)

  • Continued voluntary collaboration with the DREAM:Lab to finalize research validation for the JPDC 2025 publication
  • Authored the in-depth system architecture and scalability analysis for Flotilla, leading to its acceptance in the Journal of Parallel and Distributed Computing (JPDC)
  • Orchestrated containerized deployments using custom Docker images across distributed clusters to validate system performance on 1000+ concurrent clients

Research Associate (DREAM:Lab)

Responsibilities include:

  • Built an asynchronous federated learning framework (Flotilla) in Python, optimized for edge hardware deployment
  • Implemented server and client sides using MQTT and gRPC for efficient message passing and coordination in federated learning
  • Designed a custom Redis-based state store with checkpointing to enable recovery from full server failures without data loss or disruption
  • Integrated client selection and aggregation strategies from current research for performance, accuracy, and turnaround optimization
  • Collaborated with PhD students under Prof. Manik Gupta (BITS Pilani) and Prof. Yogesh Simmhan (IISc) to ensure Flotilla’s scalability and reliability
  • Configured and managed an 80+ node edge cluster (Nvidia Jetsons, Raspberry Pis), supporting lab infrastructure and projects including Flotilla

Volunteering:

  • Senior Student Volunteer, Indian Institute of Science – IEEE/ACM CCGrid 2023: Co-organized a 300+ participant conference; coordinated 3 poster sessions and assisted keynote speakers and faculty
  • Student Volunteer, IISc Open Day 2023: Coordinated presentation sessions for DREAM:Lab projects

Teaching Assistant, Data Engineering at Scale

Responsibilities include:

  • Taught a graduate-level course to a class of 40+ students, comprising topics such as HDFS, Map-Reduce, Apache Spark
  • Facilitated and led a 2-hour lab session per week, prepared and graded assignments, conducted one-on-one office hours, and conducted doubt-clearing sessions

Software Engineering Intern (Intellza)

Responsibilities include:

  • Developed Intellza, a unified data storage and analytics platform, alongside a cross-functional team
  • Developed and integrated a module to maintain and track schema changes for MongoDB on Intellza using LiquiBase
  • Created Docker images and optimized the existing images as per Docker’s recommendations, reducing the image size to 35% and improving the build times of the project’s CI/CD pipeline by 50%

Hobbies

Reading

Currently reading: A Wise Man’s Fear and Thinking, Fast and Slow

Tennis & Pickleball

Just enjoy hitting the ball around and having a good time with friends

Formula 1

Following F1 races, team strategies, and technological advancements

Home Lab

Running PiHole and PFSense, experimenting with network setups

3D Printing

Designing and printing 3D models for personal projects or prototyping

Astronomy

Stargazing and exploring celestial bodies, learning about the universe