running out of file descriptors
Version : nfs-ganesha-2.8.3 File descriptor limit : 65536 FSAL used : VFS Server is running out of file descriptor. /proc/<pid>/fd shows that. I see few .so ( shared objects ) and executables are open more than once . Few of them are open about 26 times. I have not debugged LRU yet. I think all open FDs are by MDCACHE. Question: Is it expected that MDCACHE would open the same file many times ( multiple FDs )? -alok
Depends on the client load. For NFSv4, you will get an open FD for each client process that opens/locks a file. For NFSv3, you will get a single global FD for all opens on a file, and an additional FD for each client process that locks the file. However, that single open FD will stick around for a long time, while in NFSv4, open FDs will be closed as soon as the client is done with them. If you're using NFSv3, some work was done on closing the global FDs faster, but this is not in 2,8, only in 3.4 and later. (This, of course, only applies to FSAL VFS or GPFS; other FSALs don't open system FDs for anything) Daniel On 3/13/21 10:21 PM, Alok Sinha wrote:
Version : nfs-ganesha-2.8.3 File descriptor limit : 65536 FSAL used : VFS
Server is running out of file descriptor. /proc/<pid>/fd shows that. I see few .so ( shared objects ) and executables are open more than once . Few of them are open about 26 times. I have not debugged LRU yet. I think all open FDs are by MDCACHE.
Question: Is it expected that MDCACHE would open the same file many times ( multiple FDs )?
-alok
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
Hi Daniel, We are using NFSv4. If the client locks a file and crashes. Are FD on the server reaped and at what internal? We had no clients running and all of the FDs on the server were still open. Hence the question. -alok On Mon, Mar 15, 2021 at 6:26 AM Daniel Gryniewicz <dang@redhat.com> wrote:
Depends on the client load. For NFSv4, you will get an open FD for each client process that opens/locks a file. For NFSv3, you will get a single global FD for all opens on a file, and an additional FD for each client process that locks the file. However, that single open FD will stick around for a long time, while in NFSv4, open FDs will be closed as soon as the client is done with them.
If you're using NFSv3, some work was done on closing the global FDs faster, but this is not in 2,8, only in 3.4 and later.
(This, of course, only applies to FSAL VFS or GPFS; other FSALs don't open system FDs for anything)
Daniel
On 3/13/21 10:21 PM, Alok Sinha wrote:
Version : nfs-ganesha-2.8.3 File descriptor limit : 65536 FSAL used : VFS
Server is running out of file descriptor. /proc/<pid>/fd shows that. I see few .so ( shared objects ) and executables are open more than once . Few of them are open about 26 times. I have not debugged LRU
yet.
I think all open FDs are by MDCACHE.
Question: Is it expected that MDCACHE would open the same file many times ( multiple FDs )?
-alok
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
Clients are reaped after the lease expires. This means that no traffic has been received from the client in a Lease_Lifetime (configurable in the NFSv4 block, default 60) number of seconds. A reaped client has all it's FDs closed. Daniel On 3/16/21 9:11 AM, Alok Sinha wrote:
Hi Daniel,
We are using NFSv4.
If the client locks a file and crashes. Are FD on the server reaped and at what internal? We had no clients running and all of the FDs on the server were still open. Hence the question.
-alok
On Mon, Mar 15, 2021 at 6:26 AM Daniel Gryniewicz <dang@redhat.com <mailto:dang@redhat.com>> wrote:
Depends on the client load. For NFSv4, you will get an open FD for each client process that opens/locks a file. For NFSv3, you will get a single global FD for all opens on a file, and an additional FD for each client process that locks the file. However, that single open FD will stick around for a long time, while in NFSv4, open FDs will be closed as soon as the client is done with them.
If you're using NFSv3, some work was done on closing the global FDs faster, but this is not in 2,8, only in 3.4 and later.
(This, of course, only applies to FSAL VFS or GPFS; other FSALs don't open system FDs for anything)
Daniel
On 3/13/21 10:21 PM, Alok Sinha wrote: > > Version : nfs-ganesha-2.8.3 > File descriptor limit : 65536 > FSAL used : VFS > > Server is running out of file descriptor. /proc/<pid>/fd shows that. > I see few .so ( shared objects ) and executables are open more than > once . Few of them are open about 26 times. I have not debugged LRU yet. > I think all open FDs are by MDCACHE. > > Question: Is it expected that MDCACHE would open the same file many times > ( multiple FDs )? > > -alok > > _______________________________________________ > Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> > To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> > _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org>
participants (2)
-
Alok Sinha -
Daniel Gryniewicz