Is it possible to force nfs-ganesha to close all currently open fds within the cache using signal, configuration, dbus or any other way?
You can unexport all the exports. That's about it. We cannot close FDs when the client has open state, as this could result in data corruption. Daniel On 12/16/20 7:36 AM, Yoni K wrote:
Is it possible to force nfs-ganesha to close all currently open fds within the cache using signal, configuration, dbus or any other way? _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Hi Daniel, Is there away to configure Ganesha so that the LRU reaper thread would close all open FDs older the some pre-configured duration? Regards, Gershon.
No. We can't close FDs associated with client state, as that could cause data corruption. We can only close anonymous FDs, which we do in the background. Daniel On 12/16/20 1:19 PM, gershon geva wrote:
Hi Daniel,
Is there away to configure Ganesha so that the LRU reaper thread would close all open FDs older the some pre-configured duration?
Regards, Gershon. _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
No. We can't close FDs associated with client state, as that could cause data corruption. We can only close anonymous FDs, which we do in the background.
And while those aren't closed based on age of the fd, they are closed based on least recent usage so an old anonymous fd that is being continuously used will remain open. Retaining open file descriptors is good for at least some file systems. Note that with NFSv4 (or 9P), for the most part, the only open fd will correspond to an actual open file on the client. Frank
On 12/16/20 1:19 PM, gershon geva wrote:
Hi Daniel,
Is there away to configure Ganesha so that the LRU reaper thread would close all open FDs older the some pre-configured duration?
Regards, Gershon. _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
It would be great to be able to force close FDs for NFSv3. On Wed, Dec 16, 2020 at 2:10 PM Frank Filz <ffilzlnx@mindspring.com> wrote:
No. We can't close FDs associated with client state, as that could cause data corruption. We can only close anonymous FDs, which we do in the background.
And while those aren't closed based on age of the fd, they are closed based on least recent usage so an old anonymous fd that is being continuously used will remain open. Retaining open file descriptors is good for at least some file systems.
Note that with NFSv4 (or 9P), for the most part, the only open fd will correspond to an actual open file on the client.
Frank
On 12/16/20 1:19 PM, gershon geva wrote:
Hi Daniel,
Is there away to configure Ganesha so that the LRU reaper thread would close all open FDs older the some pre-configured duration?
Regards, Gershon. _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
-- Please Donate to www.wikipedia.org
What qualifies as an anonymous FD? and assuming we only refer to those FDs, is it possible to configure a more aggressive background reaping process that would close them all within a pre-configured time?
An anonymous FD is one used by NFSv3 read/write (but not lock), or explicitly as an anonymous FD in NFSv3. All normal NFSv4 I/O and all locks in both protocols need to have FDs kept open for the duration of the client operation. It's not impossible to force close anonymous FDs, but why do you want to do that? We close them in the background, and it's tunable with many option in the MDCACHE block. What behavior do you need that the current system doesn't provide? Daniel On 12/17/20 6:25 AM, gershon geva wrote:
What qualifies as an anonymous FD? and assuming we only refer to those FDs, is it possible to configure a more aggressive background reaping process that would close them all within a pre-configured time? _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Hi Daniel, We run Ganesha on top of our proprietary filesystem WekaFS. Some of our filesystem background cleanup tasks requires that all FDs on a given export be closed. Our current NFS implementation closes all open anonymous FDs after a pre-configured time period and allow those tasks to run when the NFS activity is relatively idle. We wish to have more or less the same behavior with Ganesha.
Okay, I understand the reasoning. This is inherently impossible with NFSv4, since the protocol specification itself requires that FDs remain open as long as the client is using them. Similarly, it's impossible with any kind of locking (either NFSv3 or NFSv4), since, on linux, you must keep your FD open for the lock to continue to exist. For straight, non-locking NFSv3, it's possible, but it's not currently implemented. The FD closing threads in Ganesha are not designed to close all anonymous FDs, but rather to keep the number of open anonymous FDs low. Work could be done to close them all, but no code exists currently. Daniel On 12/19/20 11:45 AM, gershon geva wrote:
Hi Daniel,
We run Ganesha on top of our proprietary filesystem WekaFS. Some of our filesystem background cleanup tasks requires that all FDs on a given export be closed. Our current NFS implementation closes all open anonymous FDs after a pre-configured time period and allow those tasks to run when the NFS activity is relatively idle. We wish to have more or less the same behavior with Ganesha. _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
I thought setting the low percentage (FD_LWMark_Percent) to zero makes it close all anonymous fds periodically. Also, Madhu from IBM has a patch with a config setting where NFS-Ganesha opens and closes temp/anonympus fds without caching them at all. You may hit a perf hit for not cashing fds based on your FSAL/fs implementation though. On Mon, Dec 21, 2020 at 7:18 PM Daniel Gryniewicz <dang@redhat.com> wrote:
Okay, I understand the reasoning. This is inherently impossible with NFSv4, since the protocol specification itself requires that FDs remain open as long as the client is using them. Similarly, it's impossible with any kind of locking (either NFSv3 or NFSv4), since, on linux, you must keep your FD open for the lock to continue to exist.
For straight, non-locking NFSv3, it's possible, but it's not currently implemented. The FD closing threads in Ganesha are not designed to close all anonymous FDs, but rather to keep the number of open anonymous FDs low. Work could be done to close them all, but no code exists currently.
Daniel
On 12/19/20 11:45 AM, gershon geva wrote:
Hi Daniel,
We run Ganesha on top of our proprietary filesystem WekaFS. Some of our filesystem background cleanup tasks requires that all FDs on a given export be closed. Our current NFS implementation closes all open anonymous FDs after a pre-configured time period and allow those tasks to run when the NFS activity is relatively idle. We wish to have more or less the same behavior with Ganesha. _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Is will try, but there's no guarantee, as FDs on entries that are ref'd will be skipped. So, ongoing traffic can mean that there's never one time when all FDs are guaranteed to be closed. In addition, it's non-deterministic, so the FS would need some way to detect that all FDs are closed, you couldn't just trigger it. Daniel On 12/22/20 12:38 AM, Malahal Naineni wrote:
I thought setting the low percentage (FD_LWMark_Percent) to zero makes it close all anonymous fds periodically. Also, Madhu from IBM has a patch with a config setting where NFS-Ganesha opens and closes temp/anonympus fds without caching them at all. You may hit a perf hit for not cashing fds based on your FSAL/fs implementation though.
On Mon, Dec 21, 2020 at 7:18 PM Daniel Gryniewicz <dang@redhat.com <mailto:dang@redhat.com>> wrote:
Okay, I understand the reasoning. This is inherently impossible with NFSv4, since the protocol specification itself requires that FDs remain open as long as the client is using them. Similarly, it's impossible with any kind of locking (either NFSv3 or NFSv4), since, on linux, you must keep your FD open for the lock to continue to exist.
For straight, non-locking NFSv3, it's possible, but it's not currently implemented. The FD closing threads in Ganesha are not designed to close all anonymous FDs, but rather to keep the number of open anonymous FDs low. Work could be done to close them all, but no code exists currently.
Daniel
On 12/19/20 11:45 AM, gershon geva wrote: > Hi Daniel, > > We run Ganesha on top of our proprietary filesystem WekaFS. Some of our filesystem background cleanup tasks requires that all FDs on a given export be closed. Our current NFS implementation closes all open anonymous FDs after a pre-configured time period and allow those tasks to run when the NFS activity is relatively idle. We wish to have more or less the same behavior with Ganesha. > _______________________________________________ > Support mailing list -- support@lists.nfs-ganesha.org <mailto:support@lists.nfs-ganesha.org> > To unsubscribe send an email to support-leave@lists.nfs-ganesha.org <mailto:support-leave@lists.nfs-ganesha.org> > _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org <mailto:support@lists.nfs-ganesha.org> To unsubscribe send an email to support-leave@lists.nfs-ganesha.org <mailto:support-leave@lists.nfs-ganesha.org>
participants (6)
-
Daniel Gryniewicz -
Frank Filz -
gershon geva -
Malahal Naineni -
Satish Chandra Kilaru -
Yoni K