NCCL
AINCCL: Nvidia's library implementing collective communication across GPUs, the layer most distributed training frameworks sit on.
NCCL: Nvidia's library implementing collective communication across GPUs, the layer most distributed training frameworks sit on.