On 12/4/18 12:27 AM, gaurav gangalwar wrote:
> Thanks for raising it on Ganesha list.
> Just want to add, once fds are depleted, they are not getting cleaned up even after hours.
> I needed to restart Ganesha process to recover from this state.

How many hours did you wait?

Currently, the cleanup is triggered after 1023 epoll wakeups.

If you stop doing anything and simply wait, that will be 8.24 hours
(1023 * 29 seconds).

Before 2.6, that was 120 seconds for 34 hours.

After all, there's no good reason to cleanup with no activity.

Also, cleanup means fd has no recv activity in __svc_params->idle_timeout.
Ganesha default is nfs_param.core_param.rpc.idle_timeout_s = 300 seconds.

If you're using a standard client doing idles, it will never cleanup.


> I am not sure if idle cleanup code is able to get rid of extra ref and release xprt.

There is no extra ref.  We had one too few refs.  That's what DanG fixed.


> I tried with this fix in cleanup code  its working for me.
> https://paste.fedoraproject.org/paste/lapz1NrOlBxS342S4Q79aw
>
As I tried to explain where s/he reported it, that isn't a fix.  It releases
some other reference still remaining, which in turn will probably lead to
referencing freed memory.

All this does is force a close(fd), which allows more connections reusing
the old fd.  Masking the symptoms.

The real underlying security issue is that a malicious adversary can run a
resource exhaustion attack.  That's what needs to be fixed.