Blogs
Articles
xDiT Principle
This is a brief introduction to the xDiT Principle.
VLLM Sourse Code Reading
vllm structure
Functional Test of Hugo
function test
All2All Communication Cost
Introduction of Transformer Family
DistriFusion
Paper reading about DistriFusion.
DeepSpeedUlysses
Paper reading of Deepseed Ulysses.
Efficient Large-Scale Language Model Training on GPU
Paper reading about Efficient Large-Scale Language Model Training on GPU Clusters.
Megatron-LM
Paper reading about Megatron-LM
Ring Attention Principle
This is a brief introduction to the Ring Attention Principle.
Comparsion of Parallelsim Metods in ViT
Paper reading of