Data Parallelism: How to Train Deep Learning Models on Multiple GPUs

Maintaining Model Accuracy when Scaling to Multiple GPUs

Nov 6, 2023, 2:30 PM

1h 30m

Description

Understand and apply key algorithmic considerations to retain accuracy when training on multiple GPUs

• Understand what might cause accuracy to decrease when parallelizing training on multiple GPUs.

• Learn and understand techniques for maintaining accuracy when scaling training to multiple GPUs.