Hello,
We are planning to enable support for NFS over RDMA in Ganesha.
There is some existing implementation in tirpc layer for RDMA, which was not in working state and not maintained any more.
We spent some time to fix this code and add missing functionalities as mentioned below,

Issue that we found and fixed:

  1. Connection management issues connect and disconnect paths. (Fixed)

  2. Static buffer management, can't scale. (Fixed)

  3. Functional issues due not compliance with RPC over RDMA RFC. (Fixed)

  4. Synchronous callbacks. (Not fixed)

  5. Protocol buffer changes for read/readdir. (Fixed)

  6. NFSv4 referral. (Client side issue, fixed in latest kernels)

  7. Callback channels using RDMA clnt. (Not fixed)

  8. ib verbs apis ibv_post_send. (Are there any issues with ib verbs apis?)

  9. Srq changes. (Not fixed)

  10. Single completion queue manager thread for all connections. (Not fixed)

  11. Per connection buffer management (Nit fixed)


We are planning to discuss it in a Ganesha community call (Tuesday, April 23rd 2024, 7.30 PM - 8.30 PM IST)
Looking for any suggestions/feedback if anyone has tried it or has expertise in this area.

Regards,
Gaurav Gangalwar