Hi,
I'm not sure my setup is "the right way", but we encounter problems on
nfs-ganesha setup with glusterfs FSAL
glusterfs version is 3.13.2-1 ( on Debian 9 )
nfs-ganesha is 2.5.3-1
We have 2 gluster nodes ( called gluster1 an gluster2 ), both with nfs-ganesha installed.
Some NFS clients connect to gluster1, others to gluster2
We are using ACL on gluster (with ext4 on the bricks) and they are activated in ganesha
conf
We had the first problem 2 days ago, on gluster1, the second yesterday on gluster2
In the minutes befores the NFS stops responding, the only suspect log I get is :
27/05/2019 17:16:00 : epoch 5cebf468 : gluster1 : ganesha.nfsd-1877[work-123]
posix_acl_2_fsal_acl :FSAL :WARN :Cannot retrieve permission set
27/05/2019 17:16:00 : epoch 5cebf468 : gluster1 : ganesha.nfsd-1877[work-123]
posix_acl_2_fsal_acl :FSAL :WARN :Cannot retrieve permission set
( and so, thousands of lines )
the same, for second problem :
28/05/2019 18:18:16 : epoch 5cecdfa1 : gluster2 : ganesha.nfsd-11510[work-229]
posix_acl_2_fsal_acl :FSAL :WARN :Cannot retrieve permission set
28/05/2019 18:18:16 : epoch 5cecdfa1 : gluster2 : ganesha.nfsd-11510[work-229]
posix_acl_2_fsal_acl :FSAL :WARN :Cannot retrieve permission set
( and so, thousands of lines )
Restart of the nfs-ganesha process seems to be enough to restore access
The ganesha.conf file is like this :
NFS_Core_Param {
#Use supplied name other tha IP In NSM operations
NSM_Use_Caller_Name = true;
#Copy lock states into "/var/lib/nfs/ganesha" dir
Clustered = false;
#Use a non-privileged port for RQuota
Rquota_Port = 875;
}
EXPORT{
Export_Id = 1 ; # Export ID unique to each export
Path = "/alternc-html"; # Path of the volume to be exported. Eg:
"/test_volume"
FSAL {
name = GLUSTER;
hostname = "127.0.0.1"; # IP of one of the nodes in the trusted
pool
volume = "alternc-html"; # Volume name. Eg:
"test_volume"
}
Access_type = RW; # Access permissions
Squash = No_root_squash; # To enable/disable root squashing
Disable_ACL = FALSE; # To enable/disable ACL
Pseudo = "/alternc-html"; # NFSv4 pseudo path for this export. Eg:
"/test_volume_pseudo"
Protocols = "3"; ##,"4" ; # NFS protocols supported
Transports = "UDP","TCP" ; # Transport protocols supported
SecType = "sys"; # Security flavors supported
}
Show replies by date
Since on the NFS clients we don't really use ACL (we connect using NFSv3, and the
extensions for Linux-nfs ACLs aren't implented in Ganesha), I switched to
Disable_ACL=TRUE.
The warning messages about premission retrieval wen away.
But we stil have problems with our NFS-Ganesha servers.
The last day, we had segmentation faults that caused the service to interrupt
Jun 17 09:06:19 gluster1 kernel: [1254485.472446] traps: ganesha.nfsd[20890] general
protection ip:5626d6a5f6a7 sp:7f4d37f65790 error:0
Jun 17 09:06:19 gluster1 kernel: [1254485.472476] in ganesha.nfsd[5626d693e000+19c000]
Jun 17 09:06:19 gluster1 systemd[1]: nfs-ganesha.service: Main process exited,
code=killed, status=11/SEGV
Jun 17 09:06:19 gluster1 systemd[1]: nfs-ganesha.service: Unit entered failed state.
Jun 18 06:37:58 gluster1 kernel: [1331984.585840] traps: ganesha.nfsd[10178] general
protection ip:55b1a4b095b5 sp:7f6532bf9f80 error:0
Jun 18 06:37:58 gluster1 kernel: [1331984.585862] in ganesha.nfsd[55b1a4a6e000+19c000]
Jun 18 06:37:58 gluster1 systemd[1]: nfs-ganesha.service: Main process exited,
code=killed, status=11/SEGV
Jun 18 06:37:58 gluster1 systemd[1]: nfs-ganesha.service: Unit entered failed state.