Blogs
Articles
astra-Sim
source code reading of astra-sim
Transformer Family
Introduction of Transformer Family
ZeRO, ZeRO-Offload, ZeRO-Infinity
Paper reading of ZeRO.
xDiT Principle
This is a brief introduction to the xDiT Principle.
VLLM Sourse Code Reading
vllm structure
Functional Test of Hugo
function test
All2All Communication Cost
Introduction of Transformer Family
DistriFusion
Paper reading about DistriFusion.
DeepSpeedUlysses
Paper reading of Deepseed Ulysses.
Efficient Large-Scale Language Model Training on GPU
Paper reading about Efficient Large-Scale Language Model Training on GPU Clusters.