dbus-send AddExport is failing with "Error org.freedesktop.DBus.Error.InvalidArgs: Error finding exports: EXPORT(Export_Id=3, ...).FSAL(Name="VFS") because No such file or directory"
Hi, I am exploring dynamic export in NFS-Ganesha using dbus-send. I could compile NFS-Ganesha with "USE_DBUS=ON" and could also start the ganesha.nfsd. The exports which were present in my config file are properly up when I started ganesha.nfsd. But, I am not able to add any new export. dbus-send command is always failing with below error [root@localhost ~]# ls -lrt /home/ total 0 drwxr-xr-x. 3 root root 36 Jun 2 21:22 ganesha_nfs_v3 drwxr-xr-x. 2 root root 6 Jun 10 18:18 ganesha_1 drwxr-xr-x. 8 root root 199 Jun 11 15:00 ganesha_nfs_v4 [root@localhost ~]# sudo dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/etc/ganesha/export.conf 'string:EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS")' Error org.freedesktop.DBus.Error.InvalidArgs: Error finding exports: EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS") because No such file or directory [root@localhost ~]# Messages in "/var/log/ganesha.log": 11/06/2020 23:44:23 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd-26823[dbus_heartbeat] gsh_export_addexport :EXPORT :CRIT :Error finding exports: EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS") because No such file or directory 11/06/2020 23:44:23 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd-26823[dbus_heartbeat] dbus_message_entrypoint :DBUS :MAJ :Method (AddExport) on (org.ganesha.nfsd.exportmgr) failed: name = (org.freedesktop.DBus.Error.InvalidArgs), message = (Error finding exports: EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS") because No such file or directory) 11/06/2020 23:46:06 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd-26823[dbus_heartbeat] nfs_health :DBUS :WARN :Health status is unhealthy. enq new: 5445, old: 5441; deq new: 5441, old: 5441 Can someone please help on this? I have followed the steps mentioned in: https://github.com/nfs-ganesha/nfs-ganesha/wiki/Dbusinterface [root@localhost ~]# ps -ef | grep nfs root 26823 1 0 23:39 ? 00:00:00 /root/build_dir_v3_stable/ganesha.nfsd -f /etc/ganesha/ganesha.conf root 26854 25589 0 23:39 pts/1 00:00:00 grep --color=auto nfs [root@localhost ~]# cat /var/log/ganesha.log | grep dbus 11/06/2020 23:39:50 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd-26823[main] nfs_Start_threads :THREAD :EVENT :gsh_dbusthread was started successfully [root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) [root@localhost ~]# [root@localhost ~]# cat /etc/ganesha/ganesha.conf ## These are core parameters that affect Ganesha as a whole. NFS_CORE_PARAM { ## Configure the protocols that Ganesha will listen for. This is a hard ## limit, as this list determines which sockets are opened. This list ## can be restricted per export, but cannot be expanded. Protocols = 3, 4; # Enable NLM (network lock manager protocol) Enable_NLM = false; # lockmgr port NLM_PORT = 45075; Enable_RQUOTA = false; } ## These are defaults for exports. They can be overridden per-export. EXPORT_DEFAULTS { ## Restrict the protocols that may use this export. This cannot allow ## access that is denied in NFS_CORE_PARAM. Protocols = 3, 4; ## Access type for clients. Default is None, so some access must be ## given either here or in the export itself. Access_Type = RW; ## Whether to squash various users. Squash = No_Root_Squash; ## Allowed security types for this export Sectype = sys,krb5,krb5i,krb5p; } ## Configure settings for the object handle cache MDCACHE { ## The point at which object cache entries will start being reused. Entries_HWMark = 100000; } ## Configure logging. Default is to log to Syslog. Basic logging can also be ## configured from the command line LOG { ## Default log level for all components Default_Log_Level = EVENT; #Default_Log_Level = DEBUG; ## Configure per-component log levels. Components { FSAL = INFO; NFS4 = EVENT; NFS3 = EVENT; } ## Where to log Facility { name = FILE; destination = "/var/log/ganesha.log"; enable = active; } } NFS_KRB5 { Active_krb5 = false; } %include "/etc/ganesha/export.conf" [root@localhost ~]# cat /etc/ganesha/export.conf ## Configure an export for some file tree EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 1; ## Exported path (mandatory) Path = /home/ganesha_nfs_v3/; ## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_nfs_v3/; Transports = "TCP"; ## Exporting FSAL FSAL { Name = VFS; } } EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 2; ## Exported path (mandatory) Path = /home/ganesha_nfs_v4/; ## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_nfs_v4/; Transports = "TCP"; ## Exporting FSAL FSAL { Name = VFS; } } Just as a data point, DisplayExport and ShowExports are running fine. [root@localhost ~]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.DisplayExport uint16:1 method return time=1591899610.781101 sender=:1.175 -> destination=:1.178 serial=623 reply_serial=2 uint16 1 string "/home/ganesha_nfs_v3" string "/home/ganesha_nfs_v3" string "" array [ ] [root@localhost ~]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.ShowExports method return time=1591899620.339869 sender=:1.175 -> destination=:1.179 serial=633 reply_serial=2 struct { uint64 1591898990 uint64 170367428 } array [ struct { uint16 1 string "/home/ganesha_nfs_v3" boolean true boolean false boolean false boolean false boolean false boolean false boolean false boolean false struct { uint64 1591899616 uint64 118026535 } } struct { uint16 2 string "/home/ganesha_nfs_v4" boolean false boolean false boolean false boolean false boolean false boolean true boolean false boolean false struct { uint64 1591899006 uint64 449489142 } } struct { uint16 0 string "/" boolean false boolean false boolean false boolean false boolean false boolean false boolean false boolean false struct { uint64 1591898990 uint64 143657141 } } ] [root@localhost ~]#
I think the problem is that you are trying to provide the export block in the dbus command when what the dbus command specifies is a file (on the Ganesha server node) that contains the export block to be added, and an expression to indicate which export block in that file to use to add from. I played with this at one time, but I don't seem to have kept notes on exactly what I did. Perhaps someone more familiar with the add export dbus interface could give more help. Optionally, another way to add exports it to edit the config file that is used at start up and then send SIGHUP to the ganesha.nfsd process. Ganesha will then re-read the config file (in it's entirety) and process LOG, EXPORT, and EXPORT_DEFAULTS blocks. It will add new exports and update existing ones. If an export is no longer in the config file, it will be removed (depending on the specifics, that may cause problems, but in simple configurations it should work). Frank
-----Original Message----- From: roshandkolhe@gmail.com [mailto:roshandkolhe@gmail.com] Sent: Thursday, June 11, 2020 11:24 AM To: support@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Support] dbus-send AddExport is failing with "Error org.freedesktop.DBus.Error.InvalidArgs: Error finding exports: EXPORT(Export_Id=3, ...).FSAL(Name="VFS") because No such file or directory"
Hi, I am exploring dynamic export in NFS-Ganesha using dbus-send. I could compile NFS-Ganesha with "USE_DBUS=ON" and could also start the ganesha.nfsd. The exports which were present in my config file are properly up when I started ganesha.nfsd. But, I am not able to add any new export. dbus-send command is always failing with below error
[root@localhost ~]# ls -lrt /home/ total 0 drwxr-xr-x. 3 root root 36 Jun 2 21:22 ganesha_nfs_v3 drwxr-xr-x. 2 root root 6 Jun 10 18:18 ganesha_1 drwxr-xr-x. 8 root root 199 Jun 11 15:00 ganesha_nfs_v4
[root@localhost ~]# sudo dbus-send --print-reply --system -- dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/etc/ganesha/export.conf 'string:EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS")' Error org.freedesktop.DBus.Error.InvalidArgs: Error finding exports: EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS") because No such file or directory [root@localhost ~]#
Messages in "/var/log/ganesha.log": 11/06/2020 23:44:23 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd- 26823[dbus_heartbeat] gsh_export_addexport :EXPORT :CRIT :Error finding exports: EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS") because No such file or directory 11/06/2020 23:44:23 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd- 26823[dbus_heartbeat] dbus_message_entrypoint :DBUS :MAJ :Method (AddExport) on (org.ganesha.nfsd.exportmgr) failed: name = (org.freedesktop.DBus.Error.InvalidArgs), message = (Error finding exports: EXPORT(Export_Id=3, Path="/home/ganesha_1/", Pseudo="/home/ganesha_1/", Transports="TCP").FSAL(Name="VFS") because No such file or directory) 11/06/2020 23:46:06 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd- 26823[dbus_heartbeat] nfs_health :DBUS :WARN :Health status is unhealthy. enq new: 5445, old: 5441; deq new: 5441, old: 5441
Can someone please help on this?
I have followed the steps mentioned in: https://github.com/nfs-ganesha/nfs- ganesha/wiki/Dbusinterface
[root@localhost ~]# ps -ef | grep nfs root 26823 1 0 23:39 ? 00:00:00 /root/build_dir_v3_stable/ganesha.nfsd -f /etc/ganesha/ganesha.conf root 26854 25589 0 23:39 pts/1 00:00:00 grep --color=auto nfs [root@localhost ~]# cat /var/log/ganesha.log | grep dbus 11/06/2020 23:39:50 : epoch 5ee2736e : localhost.localdomain : ganesha.nfsd- 26823[main] nfs_Start_threads :THREAD :EVENT :gsh_dbusthread was started successfully [root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) [root@localhost ~]#
[root@localhost ~]# cat /etc/ganesha/ganesha.conf ## These are core parameters that affect Ganesha as a whole. NFS_CORE_PARAM { ## Configure the protocols that Ganesha will listen for. This is a hard ## limit, as this list determines which sockets are opened. This list ## can be restricted per export, but cannot be expanded. Protocols = 3, 4;
# Enable NLM (network lock manager protocol) Enable_NLM = false;
# lockmgr port NLM_PORT = 45075;
Enable_RQUOTA = false; }
## These are defaults for exports. They can be overridden per-export. EXPORT_DEFAULTS { ## Restrict the protocols that may use this export. This cannot allow ## access that is denied in NFS_CORE_PARAM. Protocols = 3, 4;
## Access type for clients. Default is None, so some access must be ## given either here or in the export itself. Access_Type = RW;
## Whether to squash various users. Squash = No_Root_Squash;
## Allowed security types for this export Sectype = sys,krb5,krb5i,krb5p; }
## Configure settings for the object handle cache MDCACHE { ## The point at which object cache entries will start being reused. Entries_HWMark = 100000; }
## Configure logging. Default is to log to Syslog. Basic logging can also be ## configured from the command line LOG { ## Default log level for all components Default_Log_Level = EVENT; #Default_Log_Level = DEBUG;
## Configure per-component log levels. Components { FSAL = INFO; NFS4 = EVENT; NFS3 = EVENT; }
## Where to log Facility { name = FILE; destination = "/var/log/ganesha.log"; enable = active; } }
NFS_KRB5 { Active_krb5 = false; }
%include "/etc/ganesha/export.conf"
[root@localhost ~]# cat /etc/ganesha/export.conf ## Configure an export for some file tree EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 1;
## Exported path (mandatory) Path = /home/ganesha_nfs_v3/;
## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_nfs_v3/;
Transports = "TCP";
## Exporting FSAL FSAL { Name = VFS; } }
EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 2;
## Exported path (mandatory) Path = /home/ganesha_nfs_v4/;
## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_nfs_v4/;
Transports = "TCP";
## Exporting FSAL FSAL { Name = VFS; } }
Just as a data point, DisplayExport and ShowExports are running fine.
[root@localhost ~]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.DisplayExport uint16:1 method return time=1591899610.781101 sender=:1.175 -> destination=:1.178 serial=623 reply_serial=2 uint16 1 string "/home/ganesha_nfs_v3" string "/home/ganesha_nfs_v3" string "" array [ ] [root@localhost ~]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.ShowExports method return time=1591899620.339869 sender=:1.175 -> destination=:1.179 serial=633 reply_serial=2 struct { uint64 1591898990 uint64 170367428 } array [ struct { uint16 1 string "/home/ganesha_nfs_v3" boolean true boolean false boolean false boolean false boolean false boolean false boolean false boolean false struct { uint64 1591899616 uint64 118026535 } } struct { uint16 2 string "/home/ganesha_nfs_v4" boolean false boolean false boolean false boolean false boolean false boolean true boolean false boolean false struct { uint64 1591899006 uint64 449489142 } } struct { uint16 0 string "/" boolean false boolean false boolean false boolean false boolean false boolean false boolean false boolean false struct { uint64 1591898990 uint64 143657141 } } ] [root@localhost ~]# _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Thank you Frank for your quick reply. You are right. I was expecting dbus to modify the config file and bring up the export which was wrong assumptions from my end. When I added an EXPORT block in export.conf and used the dbus-send command later, I could add the export without any issue. [root@localhost ~]# cat /etc/ganesha/export.conf ## Configure an export for some file tree ... EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 3; ## Exported path (mandatory) Path = /home/ganesha_1/; ## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_1/; Transports = "TCP"; ## Exporting FSAL FSAL { Name = VFS; } } [root@localhost ~]# sudo dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/etc/ganesha/export.conf 'string:EXPORT(Export_Id=3)' method return time=1591935638.984428 sender=:1.175 -> destination=:1.208 serial=36413 reply_serial=2 string "1 exports added" [root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) /home/ganesha_1 (everyone) [root@localhost ~]# As you have suggested, I have tried SIGHUP as well and it is also working fine for me. [root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) /home/ganesha_1 (everyone) [root@localhost ~]# mkdir /home/ganesha_2 [root@localhost ~]# ps -ef | grep nfs | grep -v grep root 26823 1 0 Jun11 ? 00:00:49 /root/build_dir_v3_stable/ganesha.nfsd -f /etc/ganesha/ganesha.conf == Modify export.conf to add new export [root@localhost ~]# cat /etc/ganesha/export.conf ## Configure an export for some file tree ... EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 4; ## Exported path (mandatory) Path = /home/ganesha_2/; ## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_2/; Transports = "TCP"; ## Exporting FSAL FSAL { Name = VFS; } } [root@localhost ~]# kill -SIGHUP 26823 [root@localhost ~]# ps -ef | grep nfs | grep -v grep root 26823 1 0 Jun11 ? 00:00:49 /root/build_dir_v3_stable/ganesha.nfsd -f /etc/ganesha/ganesha.conf [root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) /home/ganesha_1 (everyone) /home/ganesha_2 (everyone) [root@localhost ~]# Once again, thanks a lot for your help.!!
You can also use "ganesha_mgr add_export <args>" instead of dbus-send. See documentation or "ganesha_mgr help" for further details. On Fri, Jun 12, 2020 at 6:18 PM Roshan Kolhe <roshandkolhe@gmail.com> wrote:
Thank you Frank for your quick reply.
You are right. I was expecting dbus to modify the config file and bring up the export which was wrong assumptions from my end. When I added an EXPORT block in export.conf and used the dbus-send command later, I could add the export without any issue.
[root@localhost ~]# cat /etc/ganesha/export.conf ## Configure an export for some file tree ...
EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 3;
## Exported path (mandatory) Path = /home/ganesha_1/;
## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_1/;
Transports = "TCP";
## Exporting FSAL FSAL { Name = VFS; } } [root@localhost ~]# sudo dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/etc/ganesha/export.conf 'string:EXPORT(Export_Id=3)' method return time=1591935638.984428 sender=:1.175 -> destination=:1.208 serial=36413 reply_serial=2 string "1 exports added" [root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) /home/ganesha_1 (everyone) [root@localhost ~]#
As you have suggested, I have tried SIGHUP as well and it is also working fine for me.
[root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) /home/ganesha_1 (everyone)
[root@localhost ~]# mkdir /home/ganesha_2
[root@localhost ~]# ps -ef | grep nfs | grep -v grep root 26823 1 0 Jun11 ? 00:00:49 /root/build_dir_v3_stable/ganesha.nfsd -f /etc/ganesha/ganesha.conf
== Modify export.conf to add new export [root@localhost ~]# cat /etc/ganesha/export.conf ## Configure an export for some file tree ...
EXPORT { ## Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 4;
## Exported path (mandatory) Path = /home/ganesha_2/;
## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true) Pseudo = /home/ganesha_2/;
Transports = "TCP";
## Exporting FSAL FSAL { Name = VFS; } }
[root@localhost ~]# kill -SIGHUP 26823
[root@localhost ~]# ps -ef | grep nfs | grep -v grep root 26823 1 0 Jun11 ? 00:00:49 /root/build_dir_v3_stable/ganesha.nfsd -f /etc/ganesha/ganesha.conf
[root@localhost ~]# showmount -e Export list for localhost.localdomain: /home/ganesha_nfs_v3 (everyone) /home/ganesha_nfs_v4 (everyone) /home/ganesha_1 (everyone) /home/ganesha_2 (everyone) [root@localhost ~]#
Once again, thanks a lot for your help.!! _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
When rsyncing to a nfs-ganesha exported cephfs the process hangs, and escalates into "cache pressure" of other cephfs clients[1]. When testing the rsync with more debugging on, I noticed that rsync stalled at the 'set modtime of . '[2] After restarting the nfs-ganesha and doing a few successful ls -al on the nfs mount. I was able to create this 'stall' by doing only a 'touch /var/www/cobbler/ks_mirror/CentOS7-x86_64/' on the directory. As mentioned in the post[1]. The same xlock message is in the mds log[4] the nfs-ganesha log during this touch[5]. nfs-ganesha-2.8.1.2-0.1.el7.x86_64 nfs-ganesha-ceph-2.8.1.2-0.1.el7.x86_64 ceph version 14.2.9 (581f22da52345dba46ee232b73b990f06029a2a0) nautilus (stable) [1] https://www.mail-archive.com/ceph-users@ceph.io/msg04527.html [2] recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.i686.rpm) recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.x86_64.rpm) recv_file_name(Packages/zziplib-utils-0.13.62-12.el7.x86_64.rpm) recv_file_name(Packages/centos-release-7-8.2003.0.el7.centos.x86_64.rpm) recv_file_name(Packages/lorax-19.7.26-1.el7.centos.x86_64.rpm) received 10070 names recv_file_list done recv_generator(.,0) set modtime of . to (1587553127) Wed Apr 22 12:58:47 2020 [3] 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 60.157532 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to xlock, waiting 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : client.4022217 isn't responding to mclientcaps(revoke), ino 0x100001b9177 pending pAsLsXs issued pAsLsXsFs, sent 60.158122 seconds ago [4] 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : 1 slow requests, 1 included below; oldest blocked for > 30.156793 secs 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 30.156792 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to xlock, waiting [5] 14/06/2020 14:35:23 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb14004b60 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4153280463 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14006c50, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 16063 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0x7feb2c006d70 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 3 is OP_ACCESS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Granted ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_ACCESS in position 1 = NFS4_OK, op response size is 12 total response size is 56 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0x7feb2c006da8 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 204 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4153280463 returned XPRT_IDLE 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb4c005710 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4170057679 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4c0078c0, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 21342 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa8dd60 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 1: opcode 34 is OP_SETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] fsal_check_setattr_perms :FSAL :DEBUG :Access check returned No error (Ok for owner) 14/06/2020 14:35:33 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004c80 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4045722888 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4045722888 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4062500104 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4062500104 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4079277320 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4079277320 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb5800c180 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4096054536 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb5800bdb0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4096054536 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c001060 fd 161 context 0x7feb40003150 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.153 for Program 100003, Version 4, Function 1 has xid=225276012 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb38003da0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c001060 fd 161 (::ffff:192.168.10.153:920) xid=225276012 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb48000c90 fd 160 context 0x7feb1c005c50 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.151 for Program 100003, Version 4, Function 1 has xid=937363790 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb50002630 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb48000c90 fd 160 (::ffff:192.168.10.151:998) xid=937363790 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb48005240 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4112831752 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4112831752 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb34006230 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4129608968 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4129608968 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb2c0044d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4146386184 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4146386184 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004b90 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4163163400 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4163163400 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4179940616 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4179940616 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4196717832 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4196717832 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb400008c0 fd 164 context 0x7feb5800bdb0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.152 for Program 100003, Version 4, Function 1 has xid=1500681607 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb5800c180, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb34006410 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb400008c0 fd 164 (::ffff:192.168.10.152:747) xid=1500681607 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c003690 fd 165 context 0x7feb40003150 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.111 for Program 100003, Version 4, Function 1 has xid=1915990987 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40002f50, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb48006360 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c003690 fd 165 (::ffff:192.168.10.111:977) xid=1915990987 returned XPRT_IDLE 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb58000c90 fd 152 context 0x7feb1c005c50 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.103 for Program 100003, Version 4, Function 1 has xid=113039167 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb58000c90 fd 152 (::ffff:192.168.10.103:945) xid=113039167 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb2c000c90 fd 159 context 0x7feb48005240 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.114 for Program 100003, Version 4, Function 1 has xid=3613830069 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb2c002070 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb2c000c90 fd 159 (::ffff:192.168.10.114:687) xid=3613830069 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb140008c0 fd 163 context 0x7feb34006230 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.112 for Program 100003, Version 4, Function 1 has xid=3310701715 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb480022f0 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb140008c0 fd 163 (::ffff:192.168.10.112:729) xid=3310701715 returned XPRT_IDLE 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[chunk_lru] chunk_lru_run :INODE LRU :DEBUG :After work, threadwait=90 totalwork=0 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :FD count is 0 and low water mark is 2048: not reaping. 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :After work, open_fd_count:0 count:41 fdrate:1 new_thread_wait=90 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb2c0044d0 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4186834895 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4186834895 returned XPRT_IDLE 14/06/2020 14:35:53 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration
On Sun, 2020-06-14 at 15:17 +0200, Marc Roos wrote:
When rsyncing to a nfs-ganesha exported cephfs the process hangs, and escalates into "cache pressure" of other cephfs clients[1].
When testing the rsync with more debugging on, I noticed that rsync stalled at the 'set modtime of . '[2]
After restarting the nfs-ganesha and doing a few successful ls -al on the nfs mount. I was able to create this 'stall' by doing only a 'touch /var/www/cobbler/ks_mirror/CentOS7-x86_64/' on the directory. As mentioned in the post[1]. The same xlock message is in the mds log[4] the nfs-ganesha log during this touch[5].
nfs-ganesha-2.8.1.2-0.1.el7.x86_64 nfs-ganesha-ceph-2.8.1.2-0.1.el7.x86_64
ceph version 14.2.9 (581f22da52345dba46ee232b73b990f06029a2a0) nautilus (stable)
Until recently ganesha and ceph didn't have a mechanism to allow ceph to request that ganesha shrink entries from its caches. That was added recently to mainline ceph and ganesha, but has yet to be backported everywhere yet. See: https://tracker.ceph.com/issues/12334 If you're able to test versions of libcephfs and ganesha that have this fix, we'd welcome some feedback as to whether it helps. Cheers, Jeff
[1] https://www.mail-archive.com/ceph-users@ceph.io/msg04527.html
[2] recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.i686.rpm) recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.x86_64.rpm) recv_file_name(Packages/zziplib-utils-0.13.62-12.el7.x86_64.rpm) recv_file_name(Packages/centos-release-7-8.2003.0.el7.centos.x86_64.rpm) recv_file_name(Packages/lorax-19.7.26-1.el7.centos.x86_64.rpm) received 10070 names recv_file_list done recv_generator(.,0) set modtime of . to (1587553127) Wed Apr 22 12:58:47 2020
[3] 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 60.157532 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to xlock, waiting 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : client.4022217 isn't responding to mclientcaps(revoke), ino 0x100001b9177 pending pAsLsXs issued pAsLsXsFs, sent 60.158122 seconds ago
[4] 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : 1 slow requests, 1 included below; oldest blocked for > 30.156793 secs 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 30.156792 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to xlock, waiting
[5] 14/06/2020 14:35:23 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb14004b60 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4153280463 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14006c50, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 16063 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0x7feb2c006d70 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 3 is OP_ACCESS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Granted ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_ACCESS in position 1 = NFS4_OK, op response size is 12 total response size is 56 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0x7feb2c006da8 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 204 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4153280463 returned XPRT_IDLE 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb4c005710 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4170057679 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4c0078c0, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 21342 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa8dd60 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 1: opcode 34 is OP_SETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] fsal_check_setattr_perms :FSAL :DEBUG :Access check returned No error (Ok for owner) 14/06/2020 14:35:33 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004c80 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4045722888 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4045722888 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4062500104 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4062500104 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4079277320 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4079277320 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb5800c180 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4096054536 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb5800bdb0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4096054536 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c001060 fd 161 context 0x7feb40003150 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.153 for Program 100003, Version 4, Function 1 has xid=225276012 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb38003da0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c001060 fd 161 (::ffff:192.168.10.153:920) xid=225276012 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb48000c90 fd 160 context 0x7feb1c005c50 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.151 for Program 100003, Version 4, Function 1 has xid=937363790 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb50002630 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb48000c90 fd 160 (::ffff:192.168.10.151:998) xid=937363790 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb48005240 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4112831752 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4112831752 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb34006230 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4129608968 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4129608968 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb2c0044d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4146386184 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4146386184 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004b90 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4163163400 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4163163400 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4179940616 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4179940616 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4196717832 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4196717832 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb400008c0 fd 164 context 0x7feb5800bdb0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.152 for Program 100003, Version 4, Function 1 has xid=1500681607 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb5800c180, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb34006410 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb400008c0 fd 164 (::ffff:192.168.10.152:747) xid=1500681607 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c003690 fd 165 context 0x7feb40003150 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.111 for Program 100003, Version 4, Function 1 has xid=1915990987 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40002f50, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb48006360 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c003690 fd 165 (::ffff:192.168.10.111:977) xid=1915990987 returned XPRT_IDLE 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb58000c90 fd 152 context 0x7feb1c005c50 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.103 for Program 100003, Version 4, Function 1 has xid=113039167 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb58000c90 fd 152 (::ffff:192.168.10.103:945) xid=113039167 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb2c000c90 fd 159 context 0x7feb48005240 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.114 for Program 100003, Version 4, Function 1 has xid=3613830069 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb2c002070 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb2c000c90 fd 159 (::ffff:192.168.10.114:687) xid=3613830069 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb140008c0 fd 163 context 0x7feb34006230 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.112 for Program 100003, Version 4, Function 1 has xid=3310701715 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb480022f0 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb140008c0 fd 163 (::ffff:192.168.10.112:729) xid=3310701715 returned XPRT_IDLE 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[chunk_lru] chunk_lru_run :INODE LRU :DEBUG :After work, threadwait=90 totalwork=0 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :FD count is 0 and low water mark is 2048: not reaping. 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :After work, open_fd_count:0 count:41 fdrate:1 new_thread_wait=90 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb2c0044d0 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4186834895 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4186834895 returned XPRT_IDLE 14/06/2020 14:35:53 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
-- Jeff Layton <jlayton@poochiereds.net>
On Sun, 2020-06-14 at 15:17 +0200, Marc Roos wrote:
When rsyncing to a nfs-ganesha exported cephfs the process hangs, and escalates into "cache pressure" of other cephfs clients[1].
When testing the rsync with more debugging on, I noticed that rsync stalled at the 'set modtime of . '[2]
After restarting the nfs-ganesha and doing a few successful ls -al on the nfs mount. I was able to create this 'stall' by doing only a 'touch /var/www/cobbler/ks_mirror/CentOS7-x86_64/' on the directory. As mentioned in the post[1]. The same xlock message is in the mds log[4] the nfs-ganesha log during this touch[5].
nfs-ganesha-2.8.1.2-0.1.el7.x86_64 nfs-ganesha-ceph-2.8.1.2-0.1.el7.x86_64
ceph version 14.2.9 (581f22da52345dba46ee232b73b990f06029a2a0) nautilus (stable)
Until recently ganesha and ceph didn't have a mechanism to allow ceph to request that ganesha shrink entries from its caches. That was added recently to mainline ceph and ganesha, but has yet to be backported everywhere yet. See: https://tracker.ceph.com/issues/12334 If you're able to test versions of libcephfs and ganesha that have this fix, we'd welcome some feedback as to whether it helps. Cheers, Jeff
[1] https://www.mail-archive.com/ceph-users@ceph.io/msg04527.html
[2] recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.i686.rpm) recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.x86_64.rpm) recv_file_name(Packages/zziplib-utils-0.13.62-12.el7.x86_64.rpm) recv_file_name(Packages/centos-release-7-8.2003.0.el7.centos.x86_64.rpm) recv_file_name(Packages/lorax-19.7.26-1.el7.centos.x86_64.rpm) received 10070 names recv_file_list done recv_generator(.,0) set modtime of . to (1587553127) Wed Apr 22 12:58:47 2020
[3] 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 60.157532 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to xlock, waiting 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : client.4022217 isn't responding to mclientcaps(revoke), ino 0x100001b9177 pending pAsLsXs issued pAsLsXsFs, sent 60.158122 seconds ago
[4] 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : 1 slow requests, 1 included below; oldest blocked for > 30.156793 secs 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 30.156792 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to xlock, waiting
[5] 14/06/2020 14:35:23 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb14004b60 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4153280463 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14006c50, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 16063 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0x7feb2c006d70 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 3 is OP_ACCESS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Granted ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_ACCESS in position 1 = NFS4_OK, op response size is 12 total response size is 56 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0x7feb2c006da8 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 204 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4153280463 returned XPRT_IDLE 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb4c005710 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4170057679 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4c0078c0, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 21342 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa8dd60 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 1: opcode 34 is OP_SETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] fsal_check_setattr_perms :FSAL :DEBUG :Access check returned No error (Ok for owner) 14/06/2020 14:35:33 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004c80 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4045722888 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4045722888 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4062500104 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4062500104 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4079277320 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4079277320 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb5800c180 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4096054536 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb5800bdb0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4096054536 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c001060 fd 161 context 0x7feb40003150 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.153 for Program 100003, Version 4, Function 1 has xid=225276012 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb38003da0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c001060 fd 161 (::ffff:192.168.10.153:920) xid=225276012 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb48000c90 fd 160 context 0x7feb1c005c50 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.151 for Program 100003, Version 4, Function 1 has xid=937363790 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb50002630 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb48000c90 fd 160 (::ffff:192.168.10.151:998) xid=937363790 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb48005240 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4112831752 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4112831752 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb34006230 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4129608968 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4129608968 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb2c0044d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4146386184 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4146386184 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004b90 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4163163400 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4163163400 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4179940616 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4179940616 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4196717832 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4196717832 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb400008c0 fd 164 context 0x7feb5800bdb0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.152 for Program 100003, Version 4, Function 1 has xid=1500681607 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb5800c180, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb34006410 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb400008c0 fd 164 (::ffff:192.168.10.152:747) xid=1500681607 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c003690 fd 165 context 0x7feb40003150 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.111 for Program 100003, Version 4, Function 1 has xid=1915990987 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40002f50, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb48006360 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c003690 fd 165 (::ffff:192.168.10.111:977) xid=1915990987 returned XPRT_IDLE 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb58000c90 fd 152 context 0x7feb1c005c50 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.103 for Program 100003, Version 4, Function 1 has xid=113039167 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb58000c90 fd 152 (::ffff:192.168.10.103:945) xid=113039167 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb2c000c90 fd 159 context 0x7feb48005240 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.114 for Program 100003, Version 4, Function 1 has xid=3613830069 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb2c002070 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb2c000c90 fd 159 (::ffff:192.168.10.114:687) xid=3613830069 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb140008c0 fd 163 context 0x7feb34006230 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.112 for Program 100003, Version 4, Function 1 has xid=3310701715 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb480022f0 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb140008c0 fd 163 (::ffff:192.168.10.112:729) xid=3310701715 returned XPRT_IDLE 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[chunk_lru] chunk_lru_run :INODE LRU :DEBUG :After work, threadwait=90 totalwork=0 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :FD count is 0 and low water mark is 2048: not reaping. 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :After work, open_fd_count:0 count:41 fdrate:1 new_thread_wait=90 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb2c0044d0 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2 for Program 100003, Version 4, Function 1 has xid=4186834895 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4186834895 returned XPRT_IDLE 14/06/2020 14:35:53 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
-- Jeff Layton <jlayton@redhat.com>
Hi Jeff, how did you deduct this from the log file? I can't see where the 'error' is. -----Original Message----- Subject: Re: [NFS-Ganesha-Support] bug in nfs-ganesha? and cephfs? On Sun, 2020-06-14 at 15:17 +0200, Marc Roos wrote:
When rsyncing to a nfs-ganesha exported cephfs the process hangs, and escalates into "cache pressure" of other cephfs clients[1].
When testing the rsync with more debugging on, I noticed that rsync stalled at the 'set modtime of . '[2]
After restarting the nfs-ganesha and doing a few successful ls -al on the nfs mount. I was able to create this 'stall' by doing only a 'touch /var/www/cobbler/ks_mirror/CentOS7-x86_64/' on the directory. As mentioned in the post[1]. The same xlock message is in the mds log[4] the nfs-ganesha log during this touch[5].
nfs-ganesha-2.8.1.2-0.1.el7.x86_64 nfs-ganesha-ceph-2.8.1.2-0.1.el7.x86_64
ceph version 14.2.9 (581f22da52345dba46ee232b73b990f06029a2a0) nautilus (stable)
Until recently ganesha and ceph didn't have a mechanism to allow ceph to request that ganesha shrink entries from its caches. That was added recently to mainline ceph and ganesha, but has yet to be backported everywhere yet. See: https://tracker.ceph.com/issues/12334 If you're able to test versions of libcephfs and ganesha that have this fix, we'd welcome some feedback as to whether it helps. Cheers, Jeff
[1] https://www.mail-archive.com/ceph-users@ceph.io/msg04527.html
[2] recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.i686.rpm) recv_file_name(Packages/zziplib-devel-0.13.62-12.el7.x86_64.rpm) recv_file_name(Packages/zziplib-utils-0.13.62-12.el7.x86_64.rpm)
recv_file_name(Packages/centos-release-7-8.2003.0.el7.centos.x86_64.rpm)
recv_file_name(Packages/lorax-19.7.26-1.el7.centos.x86_64.rpm) received 10070 names recv_file_list done recv_generator(.,0) set modtime of . to (1587553127) Wed Apr 22 12:58:47 2020
[3] 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 60.157532 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to
xlock, waiting 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : client.4022217 isn't responding to mclientcaps(revoke), ino 0x100001b9177 pending pAsLsXs issued pAsLsXsFs, sent 60.158122 seconds
ago
[4] 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : 1 slow requests, 1 included below; oldest blocked for > 30.156793 secs 2020-06-14 14:35:56.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : slow request 30.156792 seconds old, received at 2020-06-14 14:35:25.977231: client_request(client.4022202:52 setattr mtime=2020-06-14 14:35:25.976975 atime =2020-06-14 14:35:25.976974 #0x100001b9177 2020-06-14 14:35:25.977014 caller_uid=500, caller_gid=500{500,1,2,3,4,6,10,}) currently failed to
xlock, waiting
[5] 14/06/2020 14:35:23 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb14004b60 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2
for Program 100003, Version 4, Function 1 has xid=4153280463 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14006c50, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 16063 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0x7feb2c006d70 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 3 is OP_ACCESS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_access_op :NFS3 :DEBUG :NFS3: DEBUG: Granted ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_ACCESS in position 1 = NFS4_OK,
op response size is 12 total response size is 56 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0x7feb2c006da8 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 204 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4153280463 returned XPRT_IDLE 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb4c005710 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2
for Program 100003, Version 4, Function 1 has xid=4170057679 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4c0078c0, tag = NO TAG 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 0: opcode 22 is OP_PUTFH 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 10 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 21342 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa8dd60 type: 7, New type: 7 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] process_one_op :NFS4 :DEBUG :Request 1: opcode 34 is OP_SETATTR 14/06/2020 14:35:25 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_3] fsal_check_setattr_perms :FSAL :DEBUG :Access check returned No error (Ok for owner) 14/06/2020 14:35:33 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004c80 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4045722888 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4045722888 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4062500104 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih AVL hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] encode_fsid :NFS4 :DEBUG :fsid.major = 4294967295, fsid.minor = 4294967294 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 144 total response size is 232 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4062500104 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4079277320 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4079277320 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb5800c180 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4096054536 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb5800bdb0, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4096054536 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c001060 fd 161 context 0x7feb40003150 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.153 for Program 100003, Version 4, Function 1 has xid=225276012 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb38003da0 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c001060 fd 161 (::ffff:192.168.10.153:920) xid=225276012 returned XPRT_IDLE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb48000c90 fd 160 context 0x7feb1c005c50 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.151 for Program 100003, Version 4, Function 1 has xid=937363790 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb50002630 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:34 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb48000c90 fd 160 (::ffff:192.168.10.151:998) xid=937363790 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb48005240 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4112831752 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4112831752 returned XPRT_IDLE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb34006230 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4129608968 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:35 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4129608968 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb2c0044d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4146386184 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4146386184 returned XPRT_IDLE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb18004b90 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4163163400 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb18003de0, tag = NO TAG 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:36 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_16] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4163163400 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb24005ba0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4179940616 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb24007110, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 30 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11701 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91800 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91838 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_13] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4179940616 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] alloc_nfs_request :DISP :DEBUG :0x7feb44001060 fd 162 context 0x7feb14004b60 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_client :HT CACHE :DEBUG :client_mgr cache hit slot 0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.113 for Program 100003, Version 4, Function 1 has xid=4196717832 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 3 operations, res = 0x7feb14007e30, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb340023d0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 1: opcode 22 is OP_PUTFH 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] get_gsh_export :HT CACHE :DEBUG :export_mgr cache hit slot 35 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :fdcache MISS 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] cih_get_by_key_latch :HT CACHE :DEBUG :cih cache hit slot 11885 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] mdcache_new_entry :INODE :DEBUG :Trying to add an already existing entry. Found entry 0xa91c70 type: 7, New type: 7 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_PUTFH in position 1 = NFS4_OK, op response size is 4 total response size is 84 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] process_one_op :NFS4 :DEBUG :Request 2: opcode 9 is OP_GETATTR 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0xa91ca8 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] complete_op :NFS4 :DEBUG :Status of OP_GETATTR in position 2 = NFS4_OK, op response size is 52 total response size is 140 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_14] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb44001060 fd 162 (::ffff:192.168.10.113:967) xid=4196717832 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] alloc_nfs_request :DISP :DEBUG :0x7feb400008c0 fd 164 context 0x7feb5800bdb0 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.152 for Program 100003, Version 4, Function 1 has xid=1500681607 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb5800c180, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb34006410 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_2] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb400008c0 fd 164 (::ffff:192.168.10.152:747) xid=1500681607 returned XPRT_IDLE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] alloc_nfs_request :DISP :DEBUG :0x7feb4c003690 fd 165 context 0x7feb40003150 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.111 for Program 100003, Version 4, Function 1 has xid=1915990987 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb40002f50, tag = NO TAG 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb48006360 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:37 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_24] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb4c003690 fd 165 (::ffff:192.168.10.111:977) xid=1915990987 returned XPRT_IDLE 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] alloc_nfs_request :DISP :DEBUG :0x7feb58000c90 fd 152 context 0x7feb1c005c50 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.103 for Program 100003, Version 4, Function 1 has xid=113039167 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb1c006d40, tag = NO TAG 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:39 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_15] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb58000c90 fd 152 (::ffff:192.168.10.103:945) xid=113039167 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] alloc_nfs_request :DISP :DEBUG :0x7feb2c000c90 fd 159 context 0x7feb48005240 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.114 for Program 100003, Version 4, Function 1 has xid=3613830069 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb4800a860, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb2c002070 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_23] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb2c000c90 fd 159 (::ffff:192.168.10.114:687) xid=3613830069 returned XPRT_IDLE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] alloc_nfs_request :DISP :DEBUG :0x7feb140008c0 fd 163 context 0x7feb34006230 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.112 for Program 100003, Version 4, Function 1 has xid=3310701715 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb34005270, tag = NO TAG 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] process_one_op :NFS4 :DEBUG :Request 0: opcode 53 is OP_SEQUENCE 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] nfs4_op_sequence :SESSIONS :DEBUG :SEQUENCE session=0x7feb480022f0 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] complete_op :NFS4 :DEBUG :Status of OP_SEQUENCE in position 0 = NFS4_OK, op response size is 40 total response size is 76 14/06/2020 14:35:40 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_18] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb140008c0 fd 163 (::ffff:192.168.10.112:729) xid=3310701715 returned XPRT_IDLE 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[chunk_lru] chunk_lru_run :INODE LRU :DEBUG :After work, threadwait=90 totalwork=0 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :FD count is 0 and low water mark is 2048: not reaping. 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[cache_lru] lru_run :INODE LRU :DEBUG :After work, open_fd_count:0 count:41 fdrate:1 new_thread_wait=90 14/06/2020 14:35:43 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] alloc_nfs_request :DISP :DEBUG :0x7feb40000ce0 fd 166 context 0x7feb2c0044d0 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs_rpc_process_request :DISP :DEBUG :Request from ::ffff:192.168.10.2
for Program 100003, Version 4, Function 1 has xid=4186834895 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] dupreq_tcp_cmpf :DUPREQ :DEBUG :dupreq_tcp_cmpf 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] nfs4_Compound :NFS4 :DEBUG :COMPOUND: There are 1 operations, res = 0x7feb2c006120, tag = NO TAG 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] process_one_op :NFS4 :DEBUG :Request 0: opcode 30 is OP_RENEW 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] complete_op :NFS4 :DEBUG :Status of OP_RENEW in position 0 = NFS4_OK, op response size is 4 total response size is 40 14/06/2020 14:35:45 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[svc_25] free_nfs_request :DISP :DEBUG :SVC_DECODE on 0x7feb40000ce0 fd 166 (::ffff:192.168.10.2:736) xid=4186834895 returned XPRT_IDLE 14/06/2020 14:35:53 : epoch 5ee617d5 : c01 : ganesha.nfsd-481476[reaper] reaper_run :CLIENT ID :DEBUG :Now checking NFS4 clients for expiration _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
-- Jeff Layton <jlayton@redhat.com>
This: 2020-06-14 14:36:26.133 7fb5edd82700 0 log_channel(cluster) log [WRN] : client.4022217 isn't responding to mclientcaps(revoke), ino 0x100001b9177 pending pAsLsXs issued pAsLsXsFs, sent 60.158122 seconds ago The client is not responding to a revoke of FILE_SHARED (Fs) caps. We've seen similar symptoms in the past with rsync-style workloads. Ganesha can pin a lot of inodes, which all have to be tracked on the MDS. That said, it looks like it's just trying to revoke Fs, so it's not 100% clear that the new cache pressure callbacks would help this situation. It could be that the client is not releasing Fs for some other reason. This is an area where libcephfs has very poor visibility, unfortunately. -- Jeff On Wed, 2020-06-17 at 00:04 +0200, Marc Roos wrote:
Hi Jeff, how did you deduct this from the log file? I can't see where the 'error' is.
-----Original Message----- Subject: Re: [NFS-Ganesha-Support] bug in nfs-ganesha? and cephfs?
On Sun, 2020-06-14 at 15:17 +0200, Marc Roos wrote:
When rsyncing to a nfs-ganesha exported cephfs the process hangs, and escalates into "cache pressure" of other cephfs clients[1].
When testing the rsync with more debugging on, I noticed that rsync stalled at the 'set modtime of . '[2]
After restarting the nfs-ganesha and doing a few successful ls -al on the nfs mount. I was able to create this 'stall' by doing only a 'touch /var/www/cobbler/ks_mirror/CentOS7-x86_64/' on the directory. As mentioned in the post[1]. The same xlock message is in the mds log[4] the nfs-ganesha log during this touch[5].
nfs-ganesha-2.8.1.2-0.1.el7.x86_64 nfs-ganesha-ceph-2.8.1.2-0.1.el7.x86_64
ceph version 14.2.9 (581f22da52345dba46ee232b73b990f06029a2a0) nautilus (stable)
-- Jeff Layton <jlayton@redhat.com>
participants (7)
-
Frank Filz -
Jeff Layton -
Jeff Layton -
Malahal Naineni -
Marc Roos -
Roshan Kolhe -
roshandkolhe@gmail.com