This is on RHEL7.x?  Is selinux enabled? On both the x86_64 and on the aarch64 boxes?  What, if anything is in the /var/log/audit/audit.log? Any AVCs?

On Wed, Aug 14, 2019 at 5:16 PM Erik Jacobson <erik.jacobson@hpe.com> wrote:
Thank you - responses below:

> Checked the tcpdump in more detail. "Execute" permission seem to have got
> denied for following paths even for root user -
>
> /cm_shared/image/images_ro_nfs/rhel76-aarch64-newkernel/etc/group
>
> /cm_shared/image/images_ro_nfs/rhel76-aarch64-newkernel/usr/share/zoneinfo/America/Chicago
>
> Whereas in case of gNFS pkt trace, root user had Execute access.
>
> I am not sure why there is a difference here anf if this caused issue. Could
> you cross check the perms on both mount point and at the backend bricks
> path?


leader1 (ganesha in question) has that gluster mounted under
/opt/clmgr/shared_storage,so:


[root@leader1 ~]# ls -l /opt/clmgr/shared_storage/image/images_ro_nfs/rhel76-aarch64-newkernel/etc/group /opt/clmgr/shared_storage/image/images_ro_nfs/rhel76-aarch64-newkernel//usr/share/zoneinfo/America/Chicago
-rw-r--r-- 1 root root  718 Aug 11 12:19 /opt/clmgr/shared_storage/image/images_ro_nfs/rhel76-aarch64-newkernel/etc/group
-rw-r--r-- 2 root root 3559 May  7  2018 /opt/clmgr/shared_storage/image/images_ro_nfs/rhel76-aarch64-newkernel//usr/share/zoneinfo/America/Chicago



Here is the path on the 9 leaders (gluster 3x3):
These two files are on subvolume 1, so they appear on three of the 9
bricks and leaders (see end).



[root@indeed ~]# pdsh -w leader[1-9] ls -l /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago|dshbak
leader8: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group: No such file or directory
leader5: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group: No such file or directory
leader5: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago: No such file or directory
leader8: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago: No such file or directory
pdsh@indeed: leader5: ssh exited with exit code 2
pdsh@indeed: leader8: ssh exited with exit code 2
leader6: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group: No such file or directory
leader6: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago: No such file or directory
pdsh@indeed: leader6: ssh exited with exit code 2
leader4: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group: No such file or directory
leader4: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago: No such file or directory
pdsh@indeed: leader4: ssh exited with exit code 2
leader9: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group: No such file or directory
leader9: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago: No such file or directory
leader7: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group: No such file or directory
leader7: ls: cannot access /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago: No such file or directory
pdsh@indeed: leader9: ssh exited with exit code 2
pdsh@indeed: leader7: ssh exited with exit code 2
----------------
leader1
----------------
-rw-r--r-- 2 root root  718 Aug 11 12:19 /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group
-rw-r--r-- 3 root root 3559 May  7  2018 /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago
----------------
leader2
----------------
-rw-r--r-- 2 root root  718 Aug 11 12:19 /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group
-rw-r--r-- 3 root root 3559 May  7  2018 /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago
----------------
leader3
----------------
-rw-r--r-- 2 root root  718 Aug 11 12:19 /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64/etc/group
-rw-r--r-- 3 root root 3559 May  7  2018 /data/brick_cm_shared/image/images_ro_nfs/rhel76-aarch64//usr/share/zoneinfo/America/Chicago
_______________________________________________
Devel mailing list -- devel@lists.nfs-ganesha.org
To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org