Async request processing at the top end
I have done some work for async processing in nfs4_Compound.c and nfs_worker_thread.c, but what we need to actually be able to return from the request processing call stack without completing the request is to have async support at the RPC layer. I'm investigating that code, but it's quite a maze of twisty passages all alike. Following the code through all the function vectors is quite a journey. I'm not sure if there is stuff we can bypass so we don't need to do work inside libntirpc. Anyone who is more familiar with that code structure that has ideas, please share. Fundamentally, in order to not leave a worker thread blocked on FSAL I/O, we need to be able to return all the way up the stack without actually retiring the request. Frank
I've put some analysis in this document: https://docs.google.com/document/d/1E0LtMCxZngS4Ri0Q5ny_kAFbl4SFt20YATjfBmQF GWw/edit?usp=sharing I've given a bunch of folks write access so they can comment and add detail that makes sense. Thanks Frank From: Frank Filz [mailto:ffilzlnx@mindspring.com] Sent: Wednesday, July 25, 2018 3:54 PM To: devel@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Devel] Async request processing at the top end I have done some work for async processing in nfs4_Compound.c and nfs_worker_thread.c, but what we need to actually be able to return from the request processing call stack without completing the request is to have async support at the RPC layer. I'm investigating that code, but it's quite a maze of twisty passages all alike. Following the code through all the function vectors is quite a journey. I'm not sure if there is stuff we can bypass so we don't need to do work inside libntirpc. Anyone who is more familiar with that code structure that has ideas, please share. Fundamentally, in order to not leave a worker thread blocked on FSAL I/O, we need to be able to return all the way up the stack without actually retiring the request. Frank
On 7/25/18 7:39 PM, Frank Filz wrote:
I’ve put some analysis in this document:
https://docs.google.com/document/d/1E0LtMCxZngS4Ri0Q5ny_kAFbl4SFt20YATjfBmQF...
I’ve given a bunch of folks write access so they can comment and add detail that makes sense.
I don't see any analysis. What are you trying to accomplish? Do you have example compounds actually sent by the Linux client that need operations to be processed asynchronously? Each svc_req is already async, so the only additional async would be individual operations within a compound. So you'd need another whole additional async layer on top of the existing svc_req, with completions that trigger the svc_req completion. This isn't likely to buy you anything, other than a lot more locking and task switches. That is, slow everything down.
On 7/25/18 6:53 PM, Frank Filz wrote:
I have done some work for async processing in nfs4_Compound.c and nfs_worker_thread.c, but what we need to actually be able to return from the request processing call stack without completing the request is to have async support at the RPC layer.
We already have async support at the RPC layer.
Fundamentally, in order to not leave a worker thread blocked on FSAL I/O, we need to be able to return all the way up the stack without actually retiring the request…
The svc_req is already ref counted. If you're handing off to an async-capable FSAL, just ref it before. The top half processing will return gracefully. The bottom half will release it after sending the output. Everything is already in place.
participants (2)
-
Frank Filz -
William Allen Simpson