At the moment we are seeing these errors:
25/07/2019 19:27:59 : epoch 5d393d8c : nas2 : ganesha.nfsd-1681[cache_lru] lru_run :INODE LRU :WARN :Futility count exceeded.  Client load is opening FDs faster than the LRU thread can close them.
25/07/2019 19:28:16 : epoch 5d393d8c : nas2 : ganesha.nfsd-1681[cache_lru] lru_run :INODE LRU :WARN :Futility count exceeded.  Client load is opening FDs faster than the LRU thread can close them.
25/07/2019 19:28:34 : epoch 5d393d8c : nas2 : ganesha.nfsd-1681[svc_54] mdcache_lru_fds_available :INODE LRU :CRIT :FD Hard Limit Exceeded, waking LRU thread.
25/07/2019 19:29:02 : epoch 5d393d8c : nas : ganesha.nfsd-1681[svc_64] mdcache_lru_fds_available :INODE LRU :CRIT :FD Hard Limit Exceeded, waking LRU thread.

The system limit:

$ cat /proc/sys/fs/nr_open
2097152

And the limit for nfs-ganesha process:

$ cat /proc/1681/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             385977               385977               processes
Max open files            2097152              2097152              files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       385977               385977               signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

And the number of open files is 

# ls /proc/1681/fd | wc -w
12739

I don't see why we are hitting the FD limit as we only have 12739 FD count.

It is impacting the NFS clients right now, file creation is fine but can't open an existing file to write.

I'm using VFS FSAL, and the software versions are:

nfs-ganesha-2.7.5-1.el7.x86_64
nfs-ganesha-vfs-2.7.5-1.el7.x86_64

Thanks.

Gin