Optimizing high-speed data transfer in distributed DAQ systems with NVIDIAs GPUDirect for RDMA
by
Max Riechelmann
→
Europe/Berlin
R114 (IPE)
R114
IPE
Description
NVIDIAs recently presented GPUDirect for RDMA technology allows direct communication on the PCIe bus between NVIDIA GPUs and other devices. The ability to bypass main memory and write directly into the GPU memory is expected to increase the bandwidth. KIRO (KITs InfiniBand remote communication library) is used to provide high-performance communication for control systems at the image beamline of the ANKA synchrotron. To improve reaction time of control system and be ready for cameras with throughput of several gigabytes per second, we have modified KIRO to use GPUDirect technology. Using this approach we were able to reach throughput of 40 GBit/s. The GPUDirect technology and the updated architecture of KIRO will be presented in the talk. The achieved performance and feasability of the integration in the current workflow will be discussed.
