Hi Soumya, thanks for your help. The ganesha process is started by root, but the tar is executed by a non-root(I use qr here) user. The mount point is /home/glusterfs. Reproduce steps 1. Prepare the tar file and mount point with root user cd /tmp/perm echo 444 > big1.hdr chown qr:qr big1.hdr tar czf 444.tgz big1.hdr chown qr:qr 444.tgz mkdir /home/glusterfs/perm && chown qr:qr /home/glusterfs/perm 2. switch to non-root user: su qr 3. cd /home/glusterfs/perm 4. tar xzf /tmp/perm/444.tgz The ganesha and nfs client are on the same host. os version : CentOS Linux release 7.5.1804 (Core) kernel version: 3.10.0-693.17.1.el7.x86_64 glusterfs api : glusterfs-api-devel-4.1.5-1.el7.x86_64 ERROR LOGS - ganesha.log 01/06/2019 06:59:48 : ganesha.nfsd-7236[svc_2] glusterfs_close_my_fd :FSAL :CRIT :Error : close returns with Permission denied ERROR LOGS - ganesha-gfapi.log [2019-05-31 22:59:48.613207] E [MSGID: 114031] [client-rpc-fops_v2.c:272:client4_0_open_cbk] 0-gv0-client-0: remote operation failed. Path: /perm/big1.hdr (c591c1bf-9494-4222-8aa1-deaaf0b44c7f) [Permission denied] ERROR LOGS - export-sdb1-brick.log [2019-05-31 22:59:48.388169] W [MSGID: 113117] [posix-metadata.c:569:posix_update_utime_in_mdata] 0-gv0-posix: posix utime set mdata failed on file [2019-05-31 22:59:48.399813] W [MSGID: 113117] [posix-metadata.c:569:posix_update_utime_in_mdata] 0-gv0-posix: posix utime set mdata failed on file [函数未实现] [2019-05-31 22:59:48.403448] I [MSGID: 139001] [posix-acl.c:269:posix_acl_log_permit_denied] 0-gv0-access-control: client: CTX_ID:51affcdc-77ef-497d-aece-36a568671907-GRAPH_ID:0-PID:7236-HOST:nfs-ganesha-PC_NAME:gv0-client-0-RECON_NO:-0, gfid: c591c1bf-9494-4222-8aa1-deaaf0b44c7f, req(uid:1000,gid:1000,perm:2,ngrps:1), ctx(uid:1000,gid:1000,in-groups:1,perm:444,updated-fop:SETATTR, acl:-) [Permission denied] [2019-05-31 22:59:48.403499] E [MSGID: 115070] [server-rpc-fops_v2.c:1442:server4_open_cbk] 0-gv0-server: 124: OPEN /perm/big1.hdr (c591c1bf-9494-4222-8aa1-deaaf0b44c7f), client: CTX_ID:51affcdc-77ef-497d-aece-36a568671907-GRAPH_ID:0-PID:7236-HOST:nfs-ganesha-PC_NAME:gv0-client-0-RECON_NO:-0, error-xlator: gv0-access-control [Permission denied] -------------------------------- ----- 原始邮件 ----- 发件人：Soumya Koduri <skoduri(a)redhat.com&gt; 收件人：zhbingyin(a)sina.com, ganesha-devel <devel(a)lists.nfs-ganesha.org&gt;, Frank Filz <ffilz(a)redhat.com&gt; 主题：[NFS-Ganesha-Devel] Re: Cannot close Error with gluster FSAL 日期：2019年06月01日 01点35分 On 5/31/19 1:55 PM, Soumya Koduri wrote: > > > On 5/31/19 4:30 AM, QR wrote: >> We cannot decompress this file with gluster FSAL, but vfs FSAL and >> kernel nfs can. >> Is anyone know about this? >> Thanks in advance. >> >> [qr@nfs-ganesha perm]$ tar xzf /tmp/perm/444.tgz >> tar: big1.hdr: Cannot close: Permission denied >> tar: Exiting with failure status due to previous errors >> [qr@nfs-ganesha perm]$ tar tvf /tmp/perm/444.tgz >> -r--r--r-- qr/qr 4 2019-05-30 11:25 big1.hdr >> > > It could be similar to the issue mentioned in [1]. Will check and confirm. I couldn't reproduce this issue. What is the OS version of the server and client machines? Also please check if there are any errors in ganesha.log, ganesha-gfapi.log and brick logs. Most probably you are hitting the issues discussed in [1]. The problem is that unlike most of the other FSALs in FSAL_GLUSTER we switch to user credentials before performing any operations on the backend file system [these changes were done to be able to run nfs-ganesha by a non-root user] The side-effect is that though first time the fd is opened as part of NFSv4.x client OPEN call, NFS-ganesha server may need to re-open the file to get additional fds to perform certain other operations like COMMIT, LOCK/LEASE and glusterfs doesn't grant RW access to those fds (as expected). Not sure if there is a clean way of fixing it. In [1], the author tried to workaround the problem by switching to root user if the ganesha process ID is also root. That means this issue will still remain if the ganesha process is started by a non-root user. @Frank, any thoughts? Thanks, Soumya > > Thanks, > Soumya > > [1] https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/447012 > >> Ganesha server info >> ganesha version: a3c6fa39ce72682049391b7e094885a8c151b0c8(V2.8-rc1) >> FSAL : gluster >> nfs client info >> nfs version : nfs4.0 >> mount options : >> rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=XXX,local_lock=none,addr=YYY >> >> >> _______________________________________________ >> Devel mailing list -- devel(a)lists.nfs-ganesha.org >> To unsubscribe send an email to devel-leave(a)lists.nfs-ganesha.org >> > _______________________________________________ > Devel mailing list -- devel(a)lists.nfs-ganesha.org > To unsubscribe send an email to devel-leave(a)lists.nfs-ganesha.org _______________________________________________ Devel mailing list -- devel(a)lists.nfs-ganesha.org To unsubscribe send an email to devel-leave(a)lists.nfs-ganesha.org _______________________________________________ Devel mailing list -- devel(a)lists.nfs-ganesha.org To unsubscribe send an email to devel-leave(a)lists.nfs-ganesha.org