언어 AI (NLP)/LLM & RAG
분산/병렬 처리 (DDP/MP)
bellmake
2025. 3. 14. 17:38
DDP : Distributed Data Parallel
DMP : Distributed Model Parallel
출처 : https://youtu.be/Wer9odeBWNg
without accumulation : 순차 처리 방식
with accumulation : 병렬 처리 방식 (multi-gpu)
출처 : https://youtu.be/toUSzwR0EV8
