1:23 p.m.
Hello all
Background:
running 4.0.6 compiled from source, rpms distributed from our satellite
I'm attaching the ganesha.conf and the export in question
I've been trying to get proxy-v4 to work in a test case. Permissions of the upstream
export should be good, I'm able to access
it as limited users and root. However, when I try to access the mount (ls the mount or
stat a file under it) I get Permission
denied. Initially I thought this was due to the ol rpc 16 group limit, and I needed to
turn on Manage_Gids, however enabling
that hasn't helped. I also tried with a user with fewer than 16 groups with no
change.
I've got full_debug logs on for FSAL, NFS4, and EXPORT, but nothing in the logs is
really jumping out to me. I've made selinux
permissive for the ganesha_t type with no change.
Based on a network capture it really does seem like ganesha is the source of the denial, I
don't see any traffic to the upstream
nfs server during the denials.
Hopefully this is simply some configuration error on my part, I'd be grateful for any
pointers.
Thanks
Shane
9:58 a.m.
Sorry that's a bit jumbled, I am able to access the upstream mount without issue, the
proxied mount via ganesha is where access is denied.
1:53 p.m.
You config looks like it should work, however, do you gave ID Mapper configured identical
on all three hosts involved (client, Ganesha server, proxied server)? Could you share you
network traces (both between client and Ganesha and between Ganesha and the proxied
server)? Could you share the debug log? You could add NFS4_ACL to log the permission
checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from the
server are in the form "user@domain" or "1234". You may need to set
NFS4 { Allow_Numeric_Owners = true; } in your config if the owner/group attributes are
numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue owners in the form
"1234" instead of "user@domain" for client side GETATTR and proxy side
SETATTR (and OPEN/CREATE). There are kernel NFS parameters that control this also for
client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you
enough to go on to fix the issue if that's the case.
Frank
-----Original Message-----
From: snehring(a)iastate.edu [mailto:snehring@iastate.edu]
Sent: Tuesday, June 21, 2022 7:59 AM
To: support(a)lists.nfs-ganesha.org
Subject: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Sorry that's a bit jumbled, I am able to access the upstream mount without issue,
the proxied mount via ganesha is where access is denied.
_______________________________________________
Support mailing list -- support(a)lists.nfs-ganesha.org To unsubscribe send an
email to support-leave(a)lists.nfs-ganesha.org
2:40 p.m.
Thank you Frank, the additional acl debug logging might've given me a direction to
go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested
ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx)
ACL(list_dir,add_file,execute,add_subdirectory,delete_child)
[svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file uid=0, file gid=
0, user uid=423245, user gid= 101,
access_type=0X7000000
[svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access Type=0X7000000
Allowed=0X0 Denied=0X7000000 DENIED
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Supported
ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've got
these trees setup with perms delegated
primarily with posix acls on the underlying filesystem instead of traditional owner/group
perms.
In the meantime I'll look at the captures and getting them filtered and investigate id
mapping.
Shane
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
You config looks like it should work, however, do you gave ID Mapper
configured identical on all three hosts involved (client,
Ganesha server, proxied server)? Could you share you network traces (both between client
and Ganesha and between Ganesha and
the proxied server)? Could you share the debug log? You could add NFS4_ACL to log the
permission checking which might help.
On the idmapper stuff, you need to determine if the owner/group attributes coming from
the server are in the form
"user@domain" or "1234". You may need to set NFS4 {
Allow_Numeric_Owners = true; } in your config if the owner/group
attributes are numeric. The parameter Only_Numeric_Owners will cause Ganesha to issue
owners in the form "1234" instead of
"user@domain" for client side GETATTR and proxy side SETATTR (and OPEN/CREATE).
There are kernel NFS parameters that control
this also for client and I believe the server.
I'm going to bet that idmapping issues are the cause here. I hope the above gives you
enough to go on to fix the issue if
that's the case.
Frank
3:19 p.m.
I'm not sure what PROXY_V4 does with ACLs, though it seems like it should support
them. If so, it supports NFSv4 ACLs. The proxied server would have to offer NFSv4 ACLs. If
the proxied server is Linux, it will translate POSIX ACLs to NFSv4 ACLs. If the proxied
server is Ganesha, it would do the same for FSALs that support POSIX ACLs (not FSAL_VFS
yet).
The call to fsal_check_access_no_acl() in the log indicates Ganesha is doing mode bit
permission checking. The log indicates the permissions are 0770, so the other permissions
are being used. That strongly suggests idmapping is failing somewhere, check out the
various things I pointed to in the previous e-mail.
Frank
-----Original Message-----
From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
Sent: Tuesday, June 21, 2022 12:41 PM
To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
Thank you Frank, the additional acl debug logging might've given me a direction
to go:
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested
ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx)
ACL(list_dir,add_file,execute,add_subdirectory,delete_child)
[svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file
uid=0, file gid= 0, user uid=423245, user gid= 101,
access_type=0X7000000
[svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access
Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7]
nfs_access_op :NFS4 ACL :DEBUG :Supported
ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
[svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
Does ganesha interpret posix acls for the purpose of determining access? I've
got these trees setup with perms delegated primarily with posix acls on the
underlying filesystem instead of traditional owner/group perms.
In the meantime I'll look at the captures and getting them filtered and
investigate id mapping.
Shane
On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
> You config looks like it should work, however, do you gave ID Mapper
> configured identical on all three hosts involved (client, Ganesha
> server, proxied server)? Could you share you network traces (both between
client and Ganesha and between Ganesha and the proxied server)? Could you
share the debug log? You could add NFS4_ACL to log the permission checking
which might help.
>
> On the idmapper stuff, you need to determine if the owner/group
> attributes coming from the server are in the form "user@domain" or
> "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; } in
> your config if the owner/group attributes are numeric. The parameter
> Only_Numeric_Owners will cause Ganesha to issue owners in the form "1234"
instead of "user@domain" for client side GETATTR and proxy side SETATTR (and
OPEN/CREATE). There are kernel NFS parameters that control this also for client
and I believe the server.
>
> I'm going to bet that idmapping issues are the cause here. I hope the
> above gives you enough to go on to fix the issue if that's the case.
>
> Frank
>
>
5:07 p.m.
What's the best way to debug the idmapper stuff? I changed the mode of the exported
directory to 0775 which is allowing me to
access the mount now. Which would expected if the uids are wrong.
With this change I'm able to get more responses from the server, and looking at those
in the network capture it certainly looks
like the correct uids and gids are being sent from the upstream nfs server at least as far
as owner uid and group are concerned.
And once past this hurdle of accessing the mount root, permissions seem to be enforced
according to the underlying file system
acls.
I wonder if I managed to compile this without acl support, as nfs4_getfacl fails with
'Operation to request attribute not
supported' on the mount as root.
Shane
On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote:
I'm not sure what PROXY_V4 does with ACLs, though it seems like
it should support them. If so, it supports NFSv4 ACLs. The
proxied server would have to offer NFSv4 ACLs. If the proxied server is Linux, it will
translate POSIX ACLs to NFSv4 ACLs. If
the proxied server is Ganesha, it would do the same for FSALs that support POSIX ACLs
(not FSAL_VFS yet).
The call to fsal_check_access_no_acl() in the log indicates Ganesha is doing mode bit
permission checking. The log indicates
the permissions are 0770, so the other permissions are being used. That strongly suggests
idmapping is failing somewhere,
check out the various things I pointed to in the previous e-mail.
Frank
> -----Original Message-----
> From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> Sent: Tuesday, June 21, 2022 12:41 PM
> To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
>
> Thank you Frank, the additional acl debug logging might've given me a
direction
> to go:
>
> [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested
> ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx)
> ACL(list_dir,add_file,execute,add_subdirectory,delete_child)
> [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770, file
> uid=0, file gid= 0, user uid=423245, user gid= 101,
> access_type=0X7000000
> [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access
> Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7]
> nfs_access_op :NFS4 ACL :DEBUG :Supported
> ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
>
> Does ganesha interpret posix acls for the purpose of determining access? I've
> got these trees setup with perms delegated primarily with posix acls on the
> underlying filesystem instead of traditional owner/group perms.
>
> In the meantime I'll look at the captures and getting them filtered and
> investigate id mapping.
>
> Shane
>
> On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
> > You config looks like it should work, however, do you gave ID Mapper
> > configured identical on all three hosts involved (client, Ganesha
> > server, proxied server)? Could you share you network traces (both between
> client and Ganesha and between Ganesha and the proxied server)? Could you
> share the debug log? You could add NFS4_ACL to log the permission checking
> which might help.
> >
> > On the idmapper stuff, you need to determine if the owner/group
> > attributes coming from the server are in the form "user@domain" or
> > "1234". You may need to set NFS4 { Allow_Numeric_Owners = true; }
in
> > your config if the owner/group attributes are numeric. The parameter
> > Only_Numeric_Owners will cause Ganesha to issue owners in the form
"1234"
> instead of "user@domain" for client side GETATTR and proxy side SETATTR
(and
> OPEN/CREATE). There are kernel NFS parameters that control this also for client
> and I believe the server.
> >
> > I'm going to bet that idmapping issues are the cause here. I hope the
> > above gives you enough to go on to fix the issue if that's the case.
> >
> > Frank
> >
> >
6:57 p.m.
-----Original Message-----
From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
Sent: Tuesday, June 21, 2022 3:07 PM
To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
What's the best way to debug the idmapper stuff? I changed the mode of the
exported directory to 0775 which is allowing me to access the mount now.
Which would expected if the uids are wrong.
Start with is idmapd running on all nodes? Do you have the same idmapd.conf on all
systems? Also, are your user ids the same on all systems? You can look at GETATTR in the
network trace and see how the IDs are going across. You can add IDMAPPER=Full_Debug to
your LOG { COMPONENTS {} } config.
With this change I'm able to get more responses from the server,
and looking at
those in the network capture it certainly looks like the correct uids and gids are
being sent from the upstream nfs server at least as far as owner uid and group
are concerned.
And once past this hurdle of accessing the mount root, permissions seem to be
enforced according to the underlying file system acls.
The underlying filesystem will do some of the permission checking, and it's permission
checking may override Ganesha's.
I wonder if I managed to compile this without acl support, as
nfs4_getfacl fails
with 'Operation to request attribute not supported' on the mount as root.
I'll have to look at the PROXY_V4 code to see if it actually does ACLs. It might not.
Frank
Shane
On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote:
> I'm not sure what PROXY_V4 does with ACLs, though it seems like it
> should support them. If so, it supports NFSv4 ACLs. The proxied server
> would have to offer NFSv4 ACLs. If the proxied server is Linux, it will translate
POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do the
same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
>
> The call to fsal_check_access_no_acl() in the log indicates Ganesha is
> doing mode bit permission checking. The log indicates the permissions
> are 0770, so the other permissions are being used. That strongly suggests
idmapping is failing somewhere, check out the various things I pointed to in the
previous e-mail.
>
> Frank
>
> > -----Original Message-----
> > From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> > Sent: Tuesday, June 21, 2022 12:41 PM
> > To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> > Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
> >
> > Thank you Frank, the additional acl debug logging might've given me
> > a direction to go:
> >
> > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested
> > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx)
> > ACL(list_dir,add_file,execute,add_subdirectory,delete_child)
> > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770,
> > file uid=0, file gid= 0, user uid=423245, user gid= 101,
> > access_type=0X7000000
> > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access
> > Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7]
> > nfs_access_op :NFS4 ACL :DEBUG :Supported
> > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
> >
> > Does ganesha interpret posix acls for the purpose of determining
> > access? I've got these trees setup with perms delegated primarily
> > with posix acls on the underlying filesystem instead of traditional
owner/group perms.
> >
> > In the meantime I'll look at the captures and getting them filtered
> > and investigate id mapping.
> >
> > Shane
> >
> > On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
> > > You config looks like it should work, however, do you gave ID
> > > Mapper configured identical on all three hosts involved (client,
> > > Ganesha server, proxied server)? Could you share you network
> > > traces (both between
> > client and Ganesha and between Ganesha and the proxied server)?
> > Could you share the debug log? You could add NFS4_ACL to log the
> > permission checking which might help.
> > >
> > > On the idmapper stuff, you need to determine if the owner/group
> > > attributes coming from the server are in the form "user@domain"
or
> > > "1234". You may need to set NFS4 { Allow_Numeric_Owners = true;
}
> > > in your config if the owner/group attributes are numeric. The
> > > parameter Only_Numeric_Owners will cause Ganesha to issue owners in
the form "1234"
> > instead of "user@domain" for client side GETATTR and proxy side
> > SETATTR (and OPEN/CREATE). There are kernel NFS parameters that
> > control this also for client and I believe the server.
> > >
> > > I'm going to bet that idmapping issues are the cause here. I hope
> > > the above gives you enough to go on to fix the issue if that's the
case.
> > >
> > > Frank
> > >
> > >
>
>
12:18 p.m.
I'm OOO this week, but I think PROXY_V4 doesn't support ACLs correctly.
(Similar to how it *seems* like it has krb5 support from the config, but
then doesn't use it).
Shane: what's your backend NFS server? (knfsd, ganesha with VFS, netapp,
etc.)
On Tue, Jun 21, 2022 at 17:03 Frank Filz <ffilzlnx(a)mindspring.com> wrote:
> -----Original Message-----
> From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> Sent: Tuesday, June 21, 2022 3:07 PM
> To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
>
> What's the best way to debug the idmapper stuff? I changed the mode of
the
> exported directory to 0775 which is allowing me to access the mount now.
> Which would expected if the uids are wrong.
Start with is idmapd running on all nodes? Do you have the same
idmapd.conf on all systems? Also, are your user ids the same on all
systems? You can look at GETATTR in the network trace and see how the IDs
are going across. You can add IDMAPPER=Full_Debug to your LOG { COMPONENTS
{} } config.
> With this change I'm able to get more responses from the server, and
looking at
> those in the network capture it certainly looks like the correct uids
and gids are
> being sent from the upstream nfs server at least as far as owner uid and
group
> are concerned.
> And once past this hurdle of accessing the mount root, permissions seem
to be
> enforced according to the underlying file system acls.
The underlying filesystem will do some of the permission checking, and
it's permission checking may override Ganesha's.
> I wonder if I managed to compile this without acl support, as
nfs4_getfacl fails
> with 'Operation to request attribute not supported' on the mount as root.
I'll have to look at the PROXY_V4 code to see if it actually does ACLs. It
might not.
Frank
> Shane
>
> On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote:
> > I'm not sure what PROXY_V4 does with ACLs, though it seems like it
> > should support them. If so, it supports NFSv4 ACLs. The proxied server
> > would have to offer NFSv4 ACLs. If the proxied server is Linux, it
will translate
> POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do
the
> same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
> >
> > The call to fsal_check_access_no_acl() in the log indicates Ganesha is
> > doing mode bit permission checking. The log indicates the permissions
> > are 0770, so the other permissions are being used. That strongly
suggests
> idmapping is failing somewhere, check out the various things I pointed
to in the
> previous e-mail.
> >
> > Frank
> >
> > > -----Original Message-----
> > > From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> > > Sent: Tuesday, June 21, 2022 12:41 PM
> > > To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> > > Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
> > >
> > > Thank you Frank, the additional acl debug logging might've given me
> > > a direction to go:
> > >
> > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested
> > > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx)
> > > ACL(list_dir,add_file,execute,add_subdirectory,delete_child)
> > > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770,
> > > file uid=0, file gid= 0, user uid=423245, user gid= 101,
> > > access_type=0X7000000
> > > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access
> > > Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7]
> > > nfs_access_op :NFS4 ACL :DEBUG :Supported
> > > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
> > >
> > > Does ganesha interpret posix acls for the purpose of determining
> > > access? I've got these trees setup with perms delegated primarily
> > > with posix acls on the underlying filesystem instead of traditional
> owner/group perms.
> > >
> > > In the meantime I'll look at the captures and getting them filtered
> > > and investigate id mapping.
> > >
> > > Shane
> > >
> > > On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
> > > > You config looks like it should work, however, do you gave ID
> > > > Mapper configured identical on all three hosts involved (client,
> > > > Ganesha server, proxied server)? Could you share you network
> > > > traces (both between
> > > client and Ganesha and between Ganesha and the proxied server)?
> > > Could you share the debug log? You could add NFS4_ACL to log the
> > > permission checking which might help.
> > > >
> > > > On the idmapper stuff, you need to determine if the owner/group
> > > > attributes coming from the server are in the form
"user@domain" or
> > > > "1234". You may need to set NFS4 { Allow_Numeric_Owners =
true; }
> > > > in your config if the owner/group attributes are numeric. The
> > > > parameter Only_Numeric_Owners will cause Ganesha to issue owners in
> the form "1234"
> > > instead of "user@domain" for client side GETATTR and proxy side
> > > SETATTR (and OPEN/CREATE). There are kernel NFS parameters that
> > > control this also for client and I believe the server.
> > > >
> > > > I'm going to bet that idmapping issues are the cause here. I
hope
> > > > the above gives you enough to go on to fix the issue if that's
the
case.
> > > >
> > > > Frank
> > > >
> > > >
> >
> >
_______________________________________________
Support mailing list -- support(a)lists.nfs-ganesha.org
To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org
1:27 p.m.
The backend nfs is knfsd on rhel8.
On Wed, 2022-06-22 at 10:18 -0700, Solomon Boulos wrote:
I'm OOO this week, but I think PROXY_V4 doesn't support ACLs
correctly.
(Similar to how it *seems* like it has krb5 support from the config, but then
doesn't use it).
Shane: what's your backend NFS server? (knfsd, ganesha with VFS, netapp, etc.)
On Tue, Jun 21, 2022 at 17:03 Frank Filz <ffilzlnx(a)mindspring.com> wrote:
> > -----Original Message-----
> > From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> > Sent: Tuesday, June 21, 2022 3:07 PM
> > To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> > Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
> >
> > What's the best way to debug the idmapper stuff? I changed the mode of the
> > exported directory to 0775 which is allowing me to access the mount now.
> > Which would expected if the uids are wrong.
>
> Start with is idmapd running on all nodes? Do you have the same idmapd.conf
> on all systems? Also, are your user ids the same on all systems? You can
> look at GETATTR in the network trace and see how the IDs are going across.
> You can add IDMAPPER=Full_Debug to your LOG { COMPONENTS {} } config.
>
> > With this change I'm able to get more responses from the server, and
> > looking at
> > those in the network capture it certainly looks like the correct uids and
> > gids are
> > being sent from the upstream nfs server at least as far as owner uid and
> > group
> > are concerned.
> > And once past this hurdle of accessing the mount root, permissions seem to
> > be
> > enforced according to the underlying file system acls.
>
> The underlying filesystem will do some of the permission checking, and it's
> permission checking may override Ganesha's.
>
> > I wonder if I managed to compile this without acl support, as nfs4_getfacl
> > fails
> > with 'Operation to request attribute not supported' on the mount as
root.
>
> I'll have to look at the PROXY_V4 code to see if it actually does ACLs. It
> might not.
>
> Frank
>
> > Shane
> >
> > On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote:
> > > I'm not sure what PROXY_V4 does with ACLs, though it seems like it
> > > should support them. If so, it supports NFSv4 ACLs. The proxied server
> > > would have to offer NFSv4 ACLs. If the proxied server is Linux, it will
> > > translate
> > POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would do
> > the
> > same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
> > >
> > > The call to fsal_check_access_no_acl() in the log indicates Ganesha is
> > > doing mode bit permission checking. The log indicates the permissions
> > > are 0770, so the other permissions are being used. That strongly
> > > suggests
> > idmapping is failing somewhere, check out the various things I pointed to
> > in the
> > previous e-mail.
> > >
> > > Frank
> > >
> > > > -----Original Message-----
> > > > From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> > > > Sent: Tuesday, June 21, 2022 12:41 PM
> > > > To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> > > > Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
> > > >
> > > > Thank you Frank, the additional acl debug logging might've given
me
> > > > a direction to go:
> > > >
> > > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested
> > > > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx)
> > > > ACL(list_dir,add_file,execute,add_subdirectory,delete_child)
> > > > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file Mode=0770,
> > > > file uid=0, file gid= 0, user uid=423245, user gid= 101,
> > > > access_type=0X7000000
> > > > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0, Access
> > > > Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7]
> > > > nfs_access_op :NFS4 ACL :DEBUG :Supported
> > > > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted ACCESS=-,-,-,-,-,-
> > > >
> > > > Does ganesha interpret posix acls for the purpose of determining
> > > > access? I've got these trees setup with perms delegated
primarily
> > > > with posix acls on the underlying filesystem instead of traditional
> > owner/group perms.
> > > >
> > > > In the meantime I'll look at the captures and getting them
filtered
> > > > and investigate id mapping.
> > > >
> > > > Shane
> > > >
> > > > On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
> > > > > You config looks like it should work, however, do you gave ID
> > > > > Mapper configured identical on all three hosts involved
(client,
> > > > > Ganesha server, proxied server)? Could you share you network
> > > > > traces (both between
> > > > client and Ganesha and between Ganesha and the proxied server)?
> > > > Could you share the debug log? You could add NFS4_ACL to log the
> > > > permission checking which might help.
> > > > >
> > > > > On the idmapper stuff, you need to determine if the owner/group
> > > > > attributes coming from the server are in the form
"user@domain" or
> > > > > "1234". You may need to set NFS4 {
Allow_Numeric_Owners = true; }
> > > > > in your config if the owner/group attributes are numeric. The
> > > > > parameter Only_Numeric_Owners will cause Ganesha to issue owners
in
> > the form "1234"
> > > > instead of "user@domain" for client side GETATTR and proxy
side
> > > > SETATTR (and OPEN/CREATE). There are kernel NFS parameters that
> > > > control this also for client and I believe the server.
> > > > >
> > > > > I'm going to bet that idmapping issues are the cause here. I
hope
> > > > > the above gives you enough to go on to fix the issue if
that's the
> > > > > case.
> > > > >
> > > > > Frank
> > > > >
> > > > >
> > >
> > >
>
> _______________________________________________
> Support mailing list -- support(a)lists.nfs-ganesha.org
> To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org
1:33 p.m.
I think PROXY_V4 may not even be requesting ACL info from the backend. It
seems that all the getattrs in handle.c use proxy_v4_bitmap_getattr (e.g.,
https://github.com/nfs-ganesha/nfs-ganesha/blob/d6ad1b2278d912825cdee6024...
during open). That bitmap is defined as:
```
#define PROXYV4_ATTR_BIT(b) (1U << b)
#define PROXYV4_ATTR_BIT2(b) (1U << (b - 32))
static struct bitmap4 proxyv4_bitmap_getattr = {
.map[0] =
(PROXYV4_ATTR_BIT(FATTR4_SUPPORTED_ATTRS) |
PROXYV4_ATTR_BIT(FATTR4_TYPE) | PROXYV4_ATTR_BIT(FATTR4_CHANGE) |
PROXYV4_ATTR_BIT(FATTR4_SIZE) | PROXYV4_ATTR_BIT(FATTR4_FSID) |
PROXYV4_ATTR_BIT(FATTR4_FILEID)),
.map[1] =
(PROXYV4_ATTR_BIT2(FATTR4_MODE) |
PROXYV4_ATTR_BIT2(FATTR4_NUMLINKS) |
PROXYV4_ATTR_BIT2(FATTR4_OWNER) |
PROXYV4_ATTR_BIT2(FATTR4_OWNER_GROUP) |
PROXYV4_ATTR_BIT2(FATTR4_SPACE_USED) |
PROXYV4_ATTR_BIT2(FATTR4_TIME_ACCESS) |
PROXYV4_ATTR_BIT2(FATTR4_TIME_METADATA) |
PROXYV4_ATTR_BIT2(FATTR4_TIME_MODIFY) |
PROXYV4_ATTR_BIT2(FATTR4_RAWDEV)),
.bitmap4_len = 2
};
```
I don't know anything about NFSv4, but the lack of `FATTR4_ACL` there
presumably means we're not requesting it in the responses.
On Wed, Jun 22, 2022 at 11:27 AM Nehring, Shane R [LAS] <
snehring(a)iastate.edu> wrote:
The backend nfs is knfsd on rhel8.
On Wed, 2022-06-22 at 10:18 -0700, Solomon Boulos wrote:
> I'm OOO this week, but I think PROXY_V4 doesn't support ACLs correctly.
> (Similar to how it *seems* like it has krb5 support from the config, but
then
> doesn't use it).
>
> Shane: what's your backend NFS server? (knfsd, ganesha with VFS, netapp,
etc.)
>
> On Tue, Jun 21, 2022 at 17:03 Frank Filz <ffilzlnx(a)mindspring.com>
wrote:
> > > -----Original Message-----
> > > From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> > > Sent: Tuesday, June 21, 2022 3:07 PM
> > > To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> > > Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
> > >
> > > What's the best way to debug the idmapper stuff? I changed the mode
of the
> > > exported directory to 0775 which is allowing me to access the mount
now.
> > > Which would expected if the uids are wrong.
> >
> > Start with is idmapd running on all nodes? Do you have the same
idmapd.conf
> > on all systems? Also, are your user ids the same on all systems? You
can
> > look at GETATTR in the network trace and see how the IDs are going
across.
> > You can add IDMAPPER=Full_Debug to your LOG { COMPONENTS {} } config.
> >
> > > With this change I'm able to get more responses from the server, and
> > > looking at
> > > those in the network capture it certainly looks like the correct
uids and
> > > gids are
> > > being sent from the upstream nfs server at least as far as owner uid
and
> > > group
> > > are concerned.
> > > And once past this hurdle of accessing the mount root, permissions
seem to
> > > be
> > > enforced according to the underlying file system acls.
> >
> > The underlying filesystem will do some of the permission checking, and
it's
> > permission checking may override Ganesha's.
> >
> > > I wonder if I managed to compile this without acl support, as
nfs4_getfacl
> > > fails
> > > with 'Operation to request attribute not supported' on the mount
as
root.
> >
> > I'll have to look at the PROXY_V4 code to see if it actually does
ACLs. It
> > might not.
> >
> > Frank
> >
> > > Shane
> > >
> > > On Tue, 2022-06-21 at 13:19 -0700, Frank Filz wrote:
> > > > I'm not sure what PROXY_V4 does with ACLs, though it seems like
it
> > > > should support them. If so, it supports NFSv4 ACLs. The proxied
server
> > > > would have to offer NFSv4 ACLs. If the proxied server is Linux, it
will
> > > > translate
> > > POSIX ACLs to NFSv4 ACLs. If the proxied server is Ganesha, it would
do
> > > the
> > > same for FSALs that support POSIX ACLs (not FSAL_VFS yet).
> > > >
> > > > The call to fsal_check_access_no_acl() in the log indicates
Ganesha is
> > > > doing mode bit permission checking. The log indicates the
permissions
> > > > are 0770, so the other permissions are being used. That strongly
> > > > suggests
> > > idmapping is failing somewhere, check out the various things I
pointed to
> > > in the
> > > previous e-mail.
> > > >
> > > > Frank
> > > >
> > > > > -----Original Message-----
> > > > > From: Nehring, Shane R [LAS] [mailto:snehring@iastate.edu]
> > > > > Sent: Tuesday, June 21, 2022 12:41 PM
> > > > > To: ffilzlnx(a)mindspring.com; support(a)lists.nfs-ganesha.org
> > > > > Subject: Re: [NFS-Ganesha-Support] Re: Proxy-v4 access denied
> > > > >
> > > > > Thank you Frank, the additional acl debug logging might've
given
me
> > > > > a direction to go:
> > > > >
> > > > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Requested
> > > > > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > > > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :access_mask = mode(rwx)
> > > > > ACL(list_dir,add_file,execute,add_subdirectory,delete_child)
> > > > > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :file
Mode=0770,
> > > > > file uid=0, file gid= 0, user uid=423245, user gid= 101,
> > > > > access_type=0X7000000
> > > > > [svc_7] fsal_check_access_no_acl :NFS4 ACL :F_DBG :Mask=0X0,
Access
> > > > > Type=0X7000000 Allowed=0X0 Denied=0X7000000 DENIED [svc_7]
> > > > > nfs_access_op :NFS4 ACL :DEBUG :Supported
> > > > > ACCESS=READ,LOOKUP,MODIFY,EXTEND,DELETE,-
> > > > > [svc_7] nfs_access_op :NFS4 ACL :DEBUG :Granted
ACCESS=-,-,-,-,-,-
> > > > >
> > > > > Does ganesha interpret posix acls for the purpose of
determining
> > > > > access? I've got these trees setup with perms delegated
primarily
> > > > > with posix acls on the underlying filesystem instead of
traditional
> > > owner/group perms.
> > > > >
> > > > > In the meantime I'll look at the captures and getting them
filtered
> > > > > and investigate id mapping.
> > > > >
> > > > > Shane
> > > > >
> > > > > On Tue, 2022-06-21 at 11:53 -0700, Frank Filz wrote:
> > > > > > You config looks like it should work, however, do you gave
ID
> > > > > > Mapper configured identical on all three hosts involved
(client,
> > > > > > Ganesha server, proxied server)? Could you share you
network
> > > > > > traces (both between
> > > > > client and Ganesha and between Ganesha and the proxied server)?
> > > > > Could you share the debug log? You could add NFS4_ACL to log
the
> > > > > permission checking which might help.
> > > > > >
> > > > > > On the idmapper stuff, you need to determine if the
owner/group
> > > > > > attributes coming from the server are in the form
"user@domain"
or
> > > > > > "1234". You may need to set NFS4 {
Allow_Numeric_Owners =
true; }
> > > > > > in your config if the owner/group attributes are numeric.
The
> > > > > > parameter Only_Numeric_Owners will cause Ganesha to issue
owners in
> > > the form "1234"
> > > > > instead of "user@domain" for client side GETATTR and
proxy side
> > > > > SETATTR (and OPEN/CREATE). There are kernel NFS parameters that
> > > > > control this also for client and I believe the server.
> > > > > >
> > > > > > I'm going to bet that idmapping issues are the cause
here. I
hope
> > > > > > the above gives you enough to go on to fix the issue if
that's
the
> > > > > > case.
> > > > > >
> > > > > > Frank
> > > > > >
> > > > > >
> > > >
> > > >
> >
> > _______________________________________________
> > Support mailing list -- support(a)lists.nfs-ganesha.org
> > To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org
906
days inactive
915
days old
9 comments
5 participants
participants (5)
-
Frank Filz -
Nehring, Shane R [LAS] -
snehring＠iastate.edu
-
Solomon Boulos -
Solomon Boulos