回复：[NFS-Ganesha-Devel]_Re:_[NFS-Ganesha-Devel]回复：Re:_Cannot_close_Error_with_gluster_FSAL
The issue disappears after upgrading glusterfs-api-devel from 4.1.5 to 6.1. So there is sth wrong with glusterfs-api-devel-4.1.5. Thanks Soumya. -------------------------------- ----- 原始邮件 ----- 发件人：Soumya Koduri <skoduri@redhat.com> 收件人：zhbingyin@sina.com, ganesha-devel <devel@lists.nfs-ganesha.org>, Frank Filz <ffilz@redhat.com> 主题：[NFS-Ganesha-Devel]_Re:_[NFS-Ganesha-Devel]回复：Re:_Cannot_close_Error_with_gluster_FSAL 日期：2019年06月03日 19点46分 Hi, I retried the tests on RHEL-7.7 systems but still couldn't reproduce the issue. Could you try using latest gluster release (gluster-6)? Meanwhile one optimization we can work on is to make FSAL_GLUSTER open atomic. Currently we invoke two calls via gfapi - create (to get handle) followed by open (to get fd). By clubbing these two operations into one call, we could fix this perm issue to an extent if not fully. Thanks, Soumya On 6/1/19 4:56 AM, QR wrote:
Hi Soumya, thanks for your help.
The ganesha process is started by root, but the tar is executed by a non-root(I use qr here) user. The mount point is /home/glusterfs. Reproduce steps 1. Prepare the tar file and mount point with root user cd /tmp/perm echo 444 > big1.hdr chown qr:qr big1.hdr tar czf 444.tgz big1.hdr chown qr:qr 444.tgz mkdir /home/glusterfs/perm && chown qr:qr /home/glusterfs/perm 2. switch to non-root user: su qr 3. cd /home/glusterfs/perm 4. tar xzf /tmp/perm/444.tgz
The ganesha and nfs client are on the same host. os version : CentOS Linux release 7.5.1804 (Core) kernel version: 3.10.0-693.17.1.el7.x86_64 glusterfs api : glusterfs-api-devel-4.1.5-1.el7.x86_64
ERROR LOGS - ganesha.log 01/06/2019 06:59:48 : ganesha.nfsd-7236[svc_2] glusterfs_close_my_fd :FSAL :CRIT :Error : close returns with Permission denied ERROR LOGS - ganesha-gfapi.log [2019-05-31 22:59:48.613207] E [MSGID: 114031] [client-rpc-fops_v2.c:272:client4_0_open_cbk] 0-gv0-client-0: remote operation failed. Path: /perm/big1.hdr (c591c1bf-9494-4222-8aa1-deaaf0b44c7f) [Permission denied] ERROR LOGS - export-sdb1-brick.log [2019-05-31 22:59:48.388169] W [MSGID: 113117] [posix-metadata.c:569:posix_update_utime_in_mdata] 0-gv0-posix: posix utime set mdata failed on file [2019-05-31 22:59:48.399813] W [MSGID: 113117] [posix-metadata.c:569:posix_update_utime_in_mdata] 0-gv0-posix: posix utime set mdata failed on file [函数未实现] [2019-05-31 22:59:48.403448] I [MSGID: 139001] [posix-acl.c:269:posix_acl_log_permit_denied] 0-gv0-access-control: client: CTX_ID:51affcdc-77ef-497d-aece-36a568671907-GRAPH_ID:0-PID:7236-HOST:nfs-ganesha-PC_NAME:gv0-client-0-RECON_NO:-0, gfid: c591c1bf-9494-4222-8aa1-deaaf0b44c7f, req(uid:1000,gid:1000,perm:2,ngrps:1), ctx(uid:1000,gid:1000,in-groups:1,perm:444,updated-fop:SETATTR, acl:-) [Permission denied] [2019-05-31 22:59:48.403499] E [MSGID: 115070] [server-rpc-fops_v2.c:1442:server4_open_cbk] 0-gv0-server: 124: OPEN /perm/big1.hdr (c591c1bf-9494-4222-8aa1-deaaf0b44c7f), client: CTX_ID:51affcdc-77ef-497d-aece-36a568671907-GRAPH_ID:0-PID:7236-HOST:nfs-ganesha-PC_NAME:gv0-client-0-RECON_NO:-0, error-xlator: gv0-access-control [Permission denied]
--------------------------------
----- 原始邮件 ----- 发件人：Soumya Koduri <skoduri@redhat.com> 收件人：zhbingyin@sina.com, ganesha-devel <devel@lists.nfs-ganesha.org>, Frank Filz <ffilz@redhat.com> 主题：[NFS-Ganesha-Devel] Re: Cannot close Error with gluster FSAL 日期：2019年06月01日 01点35分
On 5/31/19 1:55 PM, Soumya Koduri wrote:
On 5/31/19 4:30 AM, QR wrote:
We cannot decompress this file with gluster FSAL, but vfs FSAL and kernel nfs can. Is anyone know about this? Thanks in advance.
[qr@nfs-ganesha perm]$ tar xzf /tmp/perm/444.tgz tar: big1.hdr: Cannot close: Permission denied tar: Exiting with failure status due to previous errors [qr@nfs-ganesha perm]$ tar tvf /tmp/perm/444.tgz -r--r--r-- qr/qr 4 2019-05-30 11:25 big1.hdr
It could be similar to the issue mentioned in [1]. Will check and
confirm. I couldn't reproduce this issue. What is the OS version of the server and client machines? Also please check if there are any errors in ganesha.log, ganesha-gfapi.log and brick logs. Most probably you are hitting the issues discussed in [1]. The problem is that unlike most of the other FSALs in FSAL_GLUSTER we switch to user credentials before performing any operations on the backend file system [these changes were done to be able to run nfs-ganesha by a non-root user] The side-effect is that though first time the fd is opened as part of NFSv4.x client OPEN call, NFS-ganesha server may need to re-open the file to get additional fds to perform certain other operations like COMMIT, LOCK/LEASE and glusterfs doesn't grant RW access to those fds (as expected). Not sure if there is a clean way of fixing it. In [1], the author tried to workaround the problem by switching to root user if the ganesha process ID is also root. That means this issue will still remain if the ganesha process is started by a non-root user. @Frank, any thoughts? Thanks, Soumya
Thanks, Soumya
[1] https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/447012
Ganesha server info ganesha version: a3c6fa39ce72682049391b7e094885a8c151b0c8(V2.8-rc1) FSAL : gluster nfs client info nfs version : nfs4.0 mount options :
rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=XXX,local_lock=none,addr=YYY
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
participants (1)
-
QR