The issue disappears after upgrading glusterfs-api-devel from 4.1.5 to 6.1. So there is sth wrong with glusterfs-api-devel-4.1.5.

Thanks Soumya.

--------------------------------

Hi,
I retried the tests on RHEL-7.7 systems but still couldn't reproduce the
issue. Could you try using latest gluster release (gluster-6)?
Meanwhile one optimization we can work on is to make FSAL_GLUSTER open
atomic. Currently we invoke two calls via gfapi - create (to get handle)
followed by open (to get fd). By clubbing these two operations into one
call, we could fix this perm issue to an extent if not fully.
Thanks,
Soumya
On 6/1/19 4:56 AM, QR wrote:
> Hi Soumya, thanks for your help.
>
> The ganesha process is started by root, but the tar is executed by a
> non-root(I use qr here) user.
> The mount point is /home/glusterfs.
> Reproduce steps
> 1. Prepare the tar file and mount point with root user
>     cd /tmp/perm
>     echo 444 > big1.hdr
>     chown qr:qr big1.hdr
>     tar czf 444.tgz big1.hdr
>     chown qr:qr 444.tgz
>     mkdir /home/glusterfs/perm && chown qr:qr /home/glusterfs/perm
> 2. switch to non-root user: su qr
> 3. cd /home/glusterfs/perm
> 4. tar xzf /tmp/perm/444.tgz
>
> The ganesha and nfs client are on the same host.
> os version    : CentOS Linux release 7.5.1804 (Core)
> kernel version: 3.10.0-693.17.1.el7.x86_64
> glusterfs api : glusterfs-api-devel-4.1.5-1.el7.x86_64
>
> ERROR LOGS - ganesha.log
> 01/06/2019 06:59:48 : ganesha.nfsd-7236[svc_2] glusterfs_close_my_fd
> :FSAL :CRIT :Error : close returns with Permission denied
> ERROR LOGS - ganesha-gfapi.log
> [2019-05-31 22:59:48.613207] E [MSGID: 114031]
> [client-rpc-fops_v2.c:272:client4_0_open_cbk] 0-gv0-client-0: remote
> operation failed. Path: /perm/big1.hdr
> (c591c1bf-9494-4222-8aa1-deaaf0b44c7f) [Permission denied]
> ERROR LOGS - export-sdb1-brick.log
> [2019-05-31 22:59:48.388169] W [MSGID: 113117]
> [posix-metadata.c:569:posix_update_utime_in_mdata] 0-gv0-posix: posix
> utime set mdata failed on file
> [2019-05-31 22:59:48.399813] W [MSGID: 113117]
> [posix-metadata.c:569:posix_update_utime_in_mdata] 0-gv0-posix: posix
> utime set mdata failed on file [函数未实现]
> [2019-05-31 22:59:48.403448] I [MSGID: 139001]
> [posix-acl.c:269:posix_acl_log_permit_denied] 0-gv0-access-control:
> client:
> CTX_ID:51affcdc-77ef-497d-aece-36a568671907-GRAPH_ID:0-PID:7236-HOST:nfs-ganesha-PC_NAME:gv0-client-0-RECON_NO:-0,
> gfid: c591c1bf-9494-4222-8aa1-deaaf0b44c7f,
> req(uid:1000,gid:1000,perm:2,ngrps:1),
> ctx(uid:1000,gid:1000,in-groups:1,perm:444,updated-fop:SETATTR, acl:-)
> [Permission denied]
> [2019-05-31 22:59:48.403499] E [MSGID: 115070]
> [server-rpc-fops_v2.c:1442:server4_open_cbk] 0-gv0-server: 124: OPEN
> /perm/big1.hdr (c591c1bf-9494-4222-8aa1-deaaf0b44c7f), client:
> CTX_ID:51affcdc-77ef-497d-aece-36a568671907-GRAPH_ID:0-PID:7236-HOST:nfs-ganesha-PC_NAME:gv0-client-0-RECON_NO:-0,
> error-xlator: gv0-access-control [Permission denied]
>
>
> --------------------------------
>
>
> ----- 原始邮件 -----
> 发件人：Soumya Koduri <skoduri@redhat.com>
> 收件人：zhbingyin@sina.com, ganesha-devel <devel@lists.nfs-ganesha.org>,
> Frank Filz <ffilz@redhat.com>
> 主题：[NFS-Ganesha-Devel] Re: Cannot close Error with gluster FSAL
> 日期：2019年06月01日 01点35分
>
>
> On 5/31/19 1:55 PM, Soumya Koduri wrote:
> >
> >
> > On 5/31/19 4:30 AM, QR wrote:
> >> We cannot decompress this file with gluster FSAL, but vfs FSAL and
> >> kernel nfs can.
> >> Is anyone know about this?
> >> Thanks in advance.
> >>
> >> [qr@nfs-ganesha perm]$ tar xzf /tmp/perm/444.tgz
> >> tar: big1.hdr: Cannot close: Permission denied
> >> tar: Exiting with failure status due to previous errors
> >> [qr@nfs-ganesha perm]$ tar tvf /tmp/perm/444.tgz
> >> -r--r--r-- qr/qr             4 2019-05-30 11:25 big1.hdr
> >>
> >
> > It could be similar to the issue mentioned in [1]. Will check and
> confirm.
> I couldn't reproduce this issue. What is the OS version of the server
> and client machines? Also please check if there are any errors in
> ganesha.log, ganesha-gfapi.log and brick logs. Most probably you are
> hitting the issues discussed in [1].
> The problem is that unlike most of the other FSALs in FSAL_GLUSTER we
> switch to user credentials before performing any operations on the
> backend file system [these changes were done to be able to run
> nfs-ganesha by a non-root user]
> The side-effect is that though first time the fd is opened as part of
> NFSv4.x client OPEN call, NFS-ganesha server may need to re-open the
> file to get additional fds to perform certain other operations like
> COMMIT, LOCK/LEASE and glusterfs doesn't grant RW access to those fds
> (as expected).
> Not sure if there is a clean way of fixing it. In [1], the author tried
> to workaround the problem by switching to root user if the ganesha
> process ID is also root. That means this issue will still remain if the
> ganesha process is started by a non-root user.
> @Frank,
> any thoughts?
> Thanks,
> Soumya
> >
> > Thanks,
> > Soumya
> >
> > [1] https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/447012
> >
> >> Ganesha server info
> >>      ganesha version: a3c6fa39ce72682049391b7e094885a8c151b0c8(V2.8-rc1)
> >>      FSAL           : gluster
> >> nfs client info
> >>      nfs version    : nfs4.0
> >>      mount options  :
> >>
> rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=XXX,local_lock=none,addr=YYY
>
> >>
> >>
> >> _______________________________________________
> >> Devel mailing list -- devel@lists.nfs-ganesha.org
> >> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
> >>
> > _______________________________________________
> > Devel mailing list -- devel@lists.nfs-ganesha.org
> > To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
> _______________________________________________
> Devel mailing list -- devel@lists.nfs-ganesha.org
> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
>
> _______________________________________________
> Devel mailing list -- devel@lists.nfs-ganesha.org
> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
>
_______________________________________________
Devel mailing list -- devel@lists.nfs-ganesha.org
To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org