Home / server / Questions / 1082543
Accepted
Grant Curell
Asked: 2021-11-04 08:53:06 +0800 CST

How does RDMA transfer between two servers without the CPU?

  • 772

I've been reading about RoCE and it talks about transferring data zero copy between the memory in one server directly to the memory of an application in another server. Most articles make a point to say this is done without involvement from the CPU. I don't understand how this is accomplished though. How does the data get from the adapter to the RAM without any processing from the CPU? How does it know where the application's data is in RAM?

memory rdma

1 Answers

  1. Best Answer

    I think you misunderstand "without CPU". It does not mean the CPU is not involved at all - it just means that the CPU is not doing the copying. It does - via drivers - the setup of the copy operation and then the hardware does the actual processing.

    Your link says so VERY explicitly - just read it:

    RDMA supports zero-copy networking by enabling the network adapter to transfer data directly from the wire

    That is the copy operation, not the setup. Someone has to tell the network adapter.

    Understand that modern networks where this really does shine are 100+ gigabit - 10 gigabyte+ - per second, often multiple lines combined, with high end these days being 400gigabit.

    That is a lot - for the CPU to read from RAM, loop over it then send it to the Adapter, when you could just tell the adapter which memory to read and send.

    But again, "no involvement" is not "no CPU at all", it is "CPU is not copying the data". It still does all the control.

    • 3