Hello all Background: running 4.0.6 compiled from source, rpms distributed from our satellite I'm attaching the ganesha.conf and the export in question I've been trying to get proxy-v4 to work in a test case. Permissions of the upstream export should be good, I'm able to access it as limited users and root. However, when I try to access the mount (ls the mount or stat a file under it) I get Permission denied. Initially I thought this was due to the ol rpc 16 group limit, and I needed to turn on Manage_Gids, however enabling that hasn't helped. I also tried with a user with fewer than 16 groups with no change. I've got full_debug logs on for FSAL, NFS4, and EXPORT, but nothing in the logs is really jumping out to me. I've made selinux permissive for the ganesha_t type with no change. Based on a network capture it really does seem like ganesha is the source of the denial, I don't see any traffic to the upstream nfs server during the denials. Hopefully this is simply some configuration error on my part, I'd be grateful for any pointers. Thanks Shane
Sorry that's a bit jumbled, I am able to access the upstream mount without issue, the proxied mount via ganesha is where access is denied.
You config looks like it should work, however, do you gave ID Mapper configured identical on all three hosts involved (client, Ganesha server, proxied server)? Could you share you network traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the permission checking which might help. On the idmapper stuff, you need to determine if the owner/group attributes coming from the server are in the form "user@domain" or "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in the form "1234" instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server. I'm going to bet that idmapping issues are the cause here. I hope the above gives you enough to go on to fix the issue if that's the case. Frank
-----Original Message----- From: snehring@iastate.edu [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 7:59 AM To: support@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Sorry that's a bit jumbled, I am able to access the upstream mount without issue, the proxied mount via ganesha is where access is denied. _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Thank you Frank, the additional acl debug logging might've given me a direction to go: [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid= 0, user uid=423245, user gid= 101, access_type=0X7000000 [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,- Does ganesha interpret posix acls for the purpose of determining access? I've got these trees setup with perms delegated primarily with posix acls on the underlying filesystem instead of traditional owner/group perms. In the meantime I'll look at the captures and getting them filtered and investigate id mapping. Shane On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
You config looks like it should work, however, do you gave ID Mapper configured identical on all three hosts involved (client, Ganesha server, proxied server)? Could you share you network traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the permission checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from the server are in the form "user@domain" or "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in the form "1234" instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you enough to go on to fix the issue if that's the case.
Frank
I'm not sure what PROXY_V4 does with ACLs, though it seems like it should support them. If so, it supports NFSv4 ACLs. The proxied server would have to offer NFSv4 ACLs. If the proxied server is Linux, it will translate POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do the same for FSALs that support POSIX ACLs (not FSAL_VFS yet). The call to fsal_check_access_no_acl() in the log indicates Ganesha is doing mode bit permission checking. The log indicates the permissions are 0770, so the other permissions are being used. That strongly suggests idmapping is failing somewhere, check out the various things I pointed to in the previous e-mail. Frank
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 12:41 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Thank you Frank, the additional acl debug logging might've given me a direction to go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid= 0, user uid=423245, user gid= 101, access_type=0X7000000 [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've got these trees setup with perms delegated primarily with posix acls on the underlying filesystem instead of traditional owner/group perms.
In the meantime I'll look at the captures and getting them filtered and investigate id mapping.
Shane
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
You config looks like it should work, however, do you gave ID Mapper configured identical on all three hosts involved (client, Ganesha server, proxied server)? Could you share you network traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the permission checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from the server are in the form "user@domain" or "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in the form "1234" instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you enough to go on to fix the issue if that's the case.
Frank
What's the best way to debug the idmapper stuff? I changed the mode of the exported directory to 0775 which is allowing me to access the mount now. Which would expected if the uids are wrong. With this change I'm able to get more responses from the server, and looking at those in the network capture it certainly looks like the correct uids and gids are being sent from the upstream nfs server at least as far as owner uid and group are concerned. And once past this hurdle of accessing the mount root, permissions seem to be enforced according to the underlying file system acls. I wonder if I managed to compile this without acl support, as nfs4_getfacl fails with 'Operation to request attribute not supported' on the mount as root. Shane On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote:
I'm not sure what PROXY_V4 does with ACLs, though it seems like it should support them. If so, it supports NFSv4 ACLs. The proxied server would have to offer NFSv4 ACLs. If the proxied server is Linux, it will translate POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do the same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
The call to fsal_check_access_no_acl() in the log indicates Ganesha is doing mode bit permission checking. The log indicates the permissions are 0770, so the other permissions are being used. That strongly suggests idmapping is failing somewhere, check out the various things I pointed to in the previous e-mail.
Frank
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 12:41 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Thank you Frank, the additional acl debug logging might've given me a direction to go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid= 0, user uid=423245, user gid= 101, access_type=0X7000000 [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've got these trees setup with perms delegated primarily with posix acls on the underlying filesystem instead of traditional owner/group perms.
In the meantime I'll look at the captures and getting them filtered and investigate id mapping.
Shane
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
You config looks like it should work, however, do you gave ID Mapper configured identical on all three hosts involved (client, Ganesha server, proxied server)? Could you share you network traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the permission checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from the server are in the form "user@domain" or "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in the form "1234" instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you enough to go on to fix the issue if that's the case.
Frank
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 3:07 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
What's the best way to debug the idmapper stuff? I changed the mode of the exported directory to 0775 which is allowing me to access the mount now. Which would expected if the uids are wrong.
Start with is idmapd running on all nodes? Do you have the same idmapd.conf on all systems? Also, are your user ids the same on all systems? You can look at GETATTR in the network trace and see how the IDs are going across. You can add IDMAPPER=Full_Debug to your LOG { COMPONENTS {} } config.
With this change I'm able to get more responses from the server, and looking at those in the network capture it certainly looks like the correct uids and gids are being sent from the upstream nfs server at least as far as owner uid and group are concerned. And once past this hurdle of accessing the mount root, permissions seem to be enforced according to the underlying file system acls.
The underlying filesystem will do some of the permission checking, and it's permission checking may override Ganesha's.
I wonder if I managed to compile this without acl support, as nfs4_getfacl fails with 'Operation to request attribute not supported' on the mount as root.
I'll have to look at the PROXY_V4 code to see if it actually does ACLs. It might not. Frank
Shane
I'm not sure what PROXY_V4 does with ACLs, though it seems like it should support them. If so, it supports NFSv4 ACLs. The proxied server would have to offer NFSv4 ACLs. If the proxied server is Linux, it will translate POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do the same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
The call to fsal_check_access_no_acl() in the log indicates Ganesha is doing mode bit permission checking. The log indicates the permissions are 0770, so the other permissions are being used. That strongly suggests idmapping is failing somewhere, check out the various things I pointed to in the
On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote: previous e-mail.
Frank
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 12:41 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Thank you Frank, the additional acl debug logging might've given me a direction to go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid= 0, user uid=423245, user gid= 101, access_type=0X7000000 [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've got these trees setup with perms delegated primarily with posix acls on the underlying filesystem instead of traditional
In the meantime I'll look at the captures and getting them filtered and investigate id mapping.
Shane
You config looks like it should work, however, do you gave ID Mapper configured identical on all three hosts involved (client, Ganesha server, proxied server)? Could you share you network traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote: permission checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from the server are in the form "user@domain" or "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in
owner/group perms. the form "1234"
instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you enough to go on to fix the issue if that's the case.
Frank
I'm OOO this week, but I think PROXY_V4 doesn't support ACLs correctly. (Similar to how it *seems* like it has krb5 support from the config, but then doesn't use it). Shane: what's your backend NFS server? (knfsd, ganesha with VFS, netapp, etc.) On Tue, Jun 21, 2022 at 17:03 Frank Filz <ffilzlnx@mindspring.com> wrote:
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 3:07 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
What's the best way to debug the idmapper stuff? I changed the mode of the exported directory to 0775 which is allowing me to access the mount now. Which would expected if the uids are wrong.
Start with is idmapd running on all nodes? Do you have the same idmapd.conf on all systems? Also, are your user ids the same on all systems? You can look at GETATTR in the network trace and see how the IDs are going across. You can add IDMAPPER=Full_Debug to your LOG { COMPONENTS {} } config.
With this change I'm able to get more responses from the server, and looking at those in the network capture it certainly looks like the correct uids and gids are being sent from the upstream nfs server at least as far as owner uid and group are concerned. And once past this hurdle of accessing the mount root, permissions seem to be enforced according to the underlying file system acls.
The underlying filesystem will do some of the permission checking, and it's permission checking may override Ganesha's.
I wonder if I managed to compile this without acl support, as nfs4_getfacl fails with 'Operation to request attribute not supported' on the mount as root.
I'll have to look at the PROXY_V4 code to see if it actually does ACLs. It might not.
Frank
Shane
I'm not sure what PROXY_V4 does with ACLs, though it seems like it should support them. If so, it supports NFSv4 ACLs. The proxied server would have to offer NFSv4 ACLs. If the proxied server is Linux, it will translate POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do
On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote: the same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
The call to fsal_check_access_no_acl() in the log indicates Ganesha is doing mode bit permission checking. The log indicates the permissions are 0770, so the other permissions are being used. That strongly
suggests idmapping is failing somewhere, check out the various things I pointed to in the previous e-mail.
Frank
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 12:41 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Thank you Frank, the additional acl debug logging might've given me a direction to go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid= 0, user uid=423245, user gid= 101, access_type=0X7000000 [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've got these trees setup with perms delegated primarily with posix acls on the underlying filesystem instead of traditional
In the meantime I'll look at the captures and getting them filtered and investigate id mapping.
Shane
You config looks like it should work, however, do you gave ID Mapper configured identical on all three hosts involved (client, Ganesha server, proxied server)? Could you share you network traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote: permission checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from the server are in the form "user@domain" or "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in
owner/group perms. the form "1234"
instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you enough to go on to fix the issue if that's the
case.
Frank
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
The backend nfs is knfsd on rhel8. On Wed, 2022-06-22 at 10:18 -0700, Solomon Boulos wrote:
I'm OOO this week, but I think PROXY_V4 doesn't support ACLs correctly. (Similar to how it *seems* like it has krb5 support from the config, but then doesn't use it).
Shane: what's your backend NFS server? (knfsd, ganesha with VFS, netapp, etc.)
On Tue, Jun 21, 2022 at 17:03 Frank Filz <ffilzlnx@mindspring.com> wrote:
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 3:07 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
What's the best way to debug the idmapper stuff? I changed the mode of the exported directory to 0775 which is allowing me to access the mount now. Which would expected if the uids are wrong.
Start with is idmapd running on all nodes? Do you have the same idmapd.conf on all systems? Also, are your user ids the same on all systems? You can look at GETATTR in the network trace and see how the IDs are going across. You can add IDMAPPER=Full_Debug to your LOG { COMPONENTS {} } config.
With this change I'm able to get more responses from the server, and looking at those in the network capture it certainly looks like the correct uids and gids are being sent from the upstream nfs server at least as far as owner uid and group are concerned. And once past this hurdle of accessing the mount root, permissions seem to be enforced according to the underlying file system acls.
The underlying filesystem will do some of the permission checking, and it's permission checking may override Ganesha's.
I wonder if I managed to compile this without acl support, as nfs4_getfacl fails with 'Operation to request attribute not supported' on the mount as root.
I'll have to look at the PROXY_V4 code to see if it actually does ACLs. It might not.
Frank
Shane
I'm not sure what PROXY_V4 does with ACLs, though it seems like it should support them. If so, it supports NFSv4 ACLs. The proxied server would have to offer NFSv4 ACLs. If the proxied server is Linux, it will translate POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do
On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote: the same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
The call to fsal_check_access_no_acl() in the log indicates Ganesha is doing mode bit permission checking. The log indicates the permissions are 0770, so the other permissions are being used. That strongly suggests
idmapping is failing somewhere, check out the various things I pointed to in the previous e-mail.
Frank
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 12:41 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Thank you Frank, the additional acl debug logging might've given me a direction to go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid= 0, user uid=423245, user gid= 101, access_type=0X7000000 [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've got these trees setup with perms delegated primarily with posix acls on the underlying filesystem instead of traditional
In the meantime I'll look at the captures and getting them filtered and investigate id mapping.
Shane
You config looks like it should work, however, do you gave ID Mapper configured identical on all three hosts involved (client, Ganesha server, proxied server)? Could you share you network traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote: permission checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from the server are in the form "user@domain" or "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in
owner/group perms. the form "1234"
instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you enough to go on to fix the issue if that's the case.
Frank
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
I think PROXY_V4 may not even be requesting ACL info from the backend. It seems that all the getattrs in handle.c use proxy_v4_bitmap_getattr (e.g., https://github.com/nfs-ganesha/nfs-ganesha/blob/d6ad1b2278d912825cdee6024246... during open). That bitmap is defined as: ``` #define PROXYV4_ATTR_BIT(b) (1U << b) #define PROXYV4_ATTR_BIT2(b) (1U << (b - 32)) static struct bitmap4 proxyv4_bitmap_getattr = { .map[0] = (PROXYV4_ATTR_BIT(FATTR4_SUPPORTED_ATTRS) | PROXYV4_ATTR_BIT(FATTR4_TYPE) | PROXYV4_ATTR_BIT(FATTR4_CHANGE) | PROXYV4_ATTR_BIT(FATTR4_SIZE) | PROXYV4_ATTR_BIT(FATTR4_FSID) | PROXYV4_ATTR_BIT(FATTR4_FILEID)), .map[1] = (PROXYV4_ATTR_BIT2(FATTR4_MODE) | PROXYV4_ATTR_BIT2(FATTR4_NUMLINKS) | PROXYV4_ATTR_BIT2(FATTR4_OWNER) | PROXYV4_ATTR_BIT2(FATTR4_OWNER_GROUP) | PROXYV4_ATTR_BIT2(FATTR4_SPACE_USED) | PROXYV4_ATTR_BIT2(FATTR4_TIME_ACCESS) | PROXYV4_ATTR_BIT2(FATTR4_TIME_METADATA) | PROXYV4_ATTR_BIT2(FATTR4_TIME_MODIFY) | PROXYV4_ATTR_BIT2(FATTR4_RAWDEV)), .bitmap4_len = 2 }; ``` I don't know anything about NFSv4, but the lack of `FATTR4_ACL` there presumably means we're not requesting it in the responses. On Wed, Jun 22, 2022 at 11:27 AM Nehring, Shane R [LAS] < snehring@iastate.edu> wrote:
The backend nfs is knfsd on rhel8.
I'm OOO this week, but I think PROXY_V4 doesn't support ACLs correctly. (Similar to how it *seems* like it has krb5 support from the config, but
doesn't use it).
Shane: what's your backend NFS server? (knfsd, ganesha with VFS, netapp, etc.)
On Tue, Jun 21, 2022 at 17:03 Frank Filz <ffilzlnx@mindspring.com> wrote:
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 3:07 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
What's the best way to debug the idmapper stuff? I changed the mode of the exported directory to 0775 which is allowing me to access the mount now. Which would expected if the uids are wrong.
Start with is idmapd running on all nodes? Do you have the same idmapd.conf on all systems? Also, are your user ids the same on all systems? You can look at GETATTR in the network trace and see how the IDs are going across. You can add IDMAPPER=Full_Debug to your LOG { COMPONENTS {} } config.
With this change I'm able to get more responses from the server, and looking at those in the network capture it certainly looks like the correct uids and gids are being sent from the upstream nfs server at least as far as owner uid and group are concerned. And once past this hurdle of accessing the mount root, permissions seem to be enforced according to the underlying file system acls.
The underlying filesystem will do some of the permission checking, and it's permission checking may override Ganesha's.
I wonder if I managed to compile this without acl support, as nfs4_getfacl fails with 'Operation to request attribute not supported' on the mount as root.
I'll have to look at the PROXY_V4 code to see if it actually does ACLs. It might not.
Frank
Shane
I'm not sure what PROXY_V4 does with ACLs, though it seems like it should support them. If so, it supports NFSv4 ACLs. The proxied server would have to offer NFSv4 ACLs. If the proxied server is Linux, it will translate POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do
On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote: the same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
The call to fsal_check_access_no_acl() in the log indicates
Ganesha is
doing mode bit permission checking. The log indicates the
are 0770, so the other permissions are being used. That strongly suggests idmapping is failing somewhere, check out the various things I
in the previous e-mail.
Frank
-----Original Message----- From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu] Sent: Tuesday, June 21, 2022 12:41 PM To: ffilzlnx@mindspring.com; support@lists.nfs-ganesha.org Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Thank you Frank, the additional acl debug logging might've given
me
a direction to go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx) ACL(list_dir,add_file,execute,add_subdirectory,delete_child) [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid= 0, user uid=423245, user gid= 101, access_type=0X7000000 [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,- [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've got these trees setup with perms delegated primarily with posix acls on the underlying filesystem instead of
owner/group perms.
In the meantime I'll look at the captures and getting them
filtered
and investigate id mapping.
Shane
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote: > You config looks like it should work, however, do you gave ID > Mapper configured identical on all three hosts involved (client, > Ganesha server, proxied server)? Could you share you network > traces (both between client and Ganesha and between Ganesha and the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the permission checking which might help. > > On the idmapper stuff, you need to determine if the owner/group > attributes coming from the server are in the form "user@domain" or > "1234". You may need to set NFS4 { Allow_Numeric_Owners =
> in your config if the owner/group attributes are numeric. The > parameter Only_Numeric_Owners will cause Ganesha to issue owners in the form "1234" instead of "user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for client and I believe the server. > > I'm going to bet that idmapping issues are the cause here. I hope > the above gives you enough to go on to fix the issue if that's
On Wed, 2022-06-22 at 10:18 -0700, Solomon Boulos wrote: then permissions pointed to traditional true; } the
> case. > > Frank > >
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
participants (5)
-
Frank Filz -
Nehring, Shane R [LAS] -
snehring＠iastate.edu
-
Solomon Boulos -
Solomon Boulos