KRB5, nfs-ganesha and directories appearing with ownership as nobody/nobody on client side.
Hey All, Wondering if there would be a cross domain issue if KRB5 is enabled on NFS Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? Users get a permission denied on clients when accessing the remote mount and while checking I see it's owned by nobody / nobody rather then their user and group. The client is on a different DNS domain and ownership comes through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm thinking. However the ONLY thing different that stands out ATM is the domains being different. Wondering if anyone ran into a similar experience and could share their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps? -- Cheers, Tom K. ------------------------------------------------------------------------------------- Living on earth is expensive, but it includes a free trip around the sun.
On 2/21/19 8:07 AM, TomK wrote:
Hey All,
Wondering if there would be a cross domain issue if KRB5 is enabled on NFS Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz?
Users get a permission denied on clients when accessing the remote mount and while checking I see it's owned by nobody / nobody rather then their user and group.
The client is on a different DNS domain and ownership comes through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm thinking. However the ONLY thing different that stands out ATM is the domains being different.
Wondering if anyone ran into a similar experience and could share their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps?
So, I believe that the clients and servers need to be in the same KRB5 domain. This would likely be 123.xyz in your case, rather than using the sub-domains. Daniel
Sent from my iPhone
On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang@redhat.com> wrote:
On 2/21/19 8:07 AM, TomK wrote: Hey All, Wondering if there would be a cross domain issue if KRB5 is enabled on NFS Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? Users get a permission denied on clients when accessing the remote mount and while checking I see it's owned by nobody / nobody rather then their user and group. The client is on a different DNS domain and ownership comes through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm thinking. However the ONLY thing different that stands out ATM is the domains being different. Wondering if anyone ran into a similar experience and could share their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps?
So, I believe that the clients and servers need to be in the same KRB5 domain. This would likely be 123.xyz in your case, rather than using the sub-domains.
K. Will have to add the realm section etc then. Need the one nfs server shared between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz is the ad dc in this case.
Daniel _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
On 2/21/2019 11:34 AM, Tom wrote:
Sent from my iPhone
On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang@redhat.com> wrote:
On 2/21/19 8:07 AM, TomK wrote: Hey All, Wondering if there would be a cross domain issue if KRB5 is enabled on NFS Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? Users get a permission denied on clients when accessing the remote mount and while checking I see it's owned by nobody / nobody rather then their user and group. The client is on a different DNS domain and ownership comes through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm thinking. However the ONLY thing different that stands out ATM is the domains being different. Wondering if anyone ran into a similar experience and could share their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps?
So, I believe that the clients and servers need to be in the same KRB5 domain. This would likely be 123.xyz in your case, rather than using the sub-domains.
K.
Will have to add the realm section etc then. Need the one nfs server shared between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz is the ad dc in this case.
Apologies, I misread I think. Thinking you mean there's only one domain configuration option in the NFS Ganesha config hence why NFS Ganesha only supports a single domain? /etc/ganesha/ganesha.conf NFSv4 { Lease_Lifetime = 20 ; IdmapConf = "/etc/idmapd.conf" ; DomainName = "b.abc.123" ; } NFS_KRB5 { PrincipalName = "nfs/nfs01.b.abc.123@B.ABC.123" ; KeytabPath = /etc/krb5.keytab ; Active_krb5 = YES ; } # cat /etc/idmapd.conf [General] Domain = b.abc.123 [Mapping] [Translation] [Static] [UMICH_SCHEMA] LDAP_server = ldap-server.local.domain.edu LDAP_base = dc=local,dc=domain,dc=edu You have new mail in /var/spool/mail/root The user is the same account on the second subdomain but the trust with the AD DC through which it's served is formed with a separate Linux KDC on that domain. Perhaps this diagram will come through correctly sam@abc.123 <-- IPA KDC 1 (a.abc.123) -> AD DC (abc.123) sam@abc.123 <-- IPA KDC 2 (b.abc.123) -----^ The NFS cluster is shared between IPA KDC 1 and 2 . So in this case, if I wanted this scenario to work, guessing I may have to create a separate NFS cluster and extend the Gluster FS over 3 new additional nodes on domain a.abc.123 to ensure the same user's files and storage is always in sync across domain / kerberos realm. Reason for all this is that we're testing options for isolating certain software / projects from other domains (keeping them contained within a specific realm). The software can modify the KDC extensively and hence why we prefer it has it's own KDC. -- Cheers, Tom K. ------------------------------------------------------------------------------------- Living on earth is expensive, but it includes a free trip around the sun.
Daniel _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
On 2/21/2019 11:34 AM, Tom wrote:
Sent from my iPhone
On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang@redhat.com> wrote:
On 2/21/19 8:07 AM, TomK wrote: Hey All, Wondering if there would be a cross domain issue if KRB5 is enabled on NFS
Users get a permission denied on clients when accessing the remote mount and while checking I see it's owned by nobody / nobody rather then their user and group. The client is on a different DNS domain and ownership comes through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm
Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? thinking. However the ONLY thing different that stands out ATM is the domains being different.
Wondering if anyone ran into a similar experience and could share their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps?
So, I believe that the clients and servers need to be in the same KRB5 domain. This would likely be 123.xyz in your case, rather than using the sub-domains.
K.
Will have to add the realm section etc then. Need the one nfs server shared between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz is the ad dc in this case.
Apologies, I misread I think. Thinking you mean there's only one domain configuration option in the NFS Ganesha config hence why NFS Ganesha only supports a single domain?
/etc/ganesha/ganesha.conf NFSv4 { Lease_Lifetime = 20 ; IdmapConf = "/etc/idmapd.conf" ; DomainName = "b.abc.123" ; } NFS_KRB5 { PrincipalName = "nfs/nfs01.b.abc.123@B.ABC.123" ; KeytabPath = /etc/krb5.keytab ; Active_krb5 = YES ; }
# cat /etc/idmapd.conf [General] Domain = b.abc.123 [Mapping] [Translation] [Static] [UMICH_SCHEMA] LDAP_server = ldap-server.local.domain.edu LDAP_base = dc=local,dc=domain,dc=edu You have new mail in /var/spool/mail/root
The user is the same account on the second subdomain but the trust with the AD DC through which it's served is formed with a separate Linux KDC on that domain.
Perhaps this diagram will come through correctly
sam@abc.123 <-- IPA KDC 1 (a.abc.123) -> AD DC (abc.123) sam@abc.123 <-- IPA KDC 2 (b.abc.123) -----^
The NFS cluster is shared between IPA KDC 1 and 2 .
So in this case, if I wanted this scenario to work, guessing I may have to create a separate NFS cluster and extend the Gluster FS over 3 new additional nodes on domain a.abc.123 to ensure the same user's files and storage is always in sync across domain / kerberos realm.
Reason for all this is that we're testing options for isolating certain software / projects from other domains (keeping them contained within a specific realm). The software can modify the KDC extensively and hence why we prefer it has it's own KDC.
Yes, I think you pretty much need two separate NFS server clusters. If you only use NFS v4.x, you might be able to have them run on the same nodes by assigning one to a different port, but then the clients need to specify the port to connect to so they get the right KDC. That Domain parameter in the NFSv4 config block I think only gets used if you don't use idmapd. I'm not sure how or if idmapd works with multiple domains (Ganesha just passes the domain through, everything is configured using idmapd.conf). But I don't think you need multiple idmap domains. So I think the issue is just going to come down to the server only has a single Kerberos service principal specified by PrincipleName in the NFS_KRB5 config block. That principal of course belongs to a single Kerberos realm. Frank
On 2/22/19 10:42 AM, Frank Filz wrote:
On 2/21/2019 11:34 AM, Tom wrote:
Sent from my iPhone
On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang@redhat.com> wrote:
On 2/21/19 8:07 AM, TomK wrote: Hey All, Wondering if there would be a cross domain issue if KRB5 is enabled on NFS
Users get a permission denied on clients when accessing the remote mount and while checking I see it's owned by nobody / nobody rather then their user and group. The client is on a different DNS domain and ownership comes through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm
Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? thinking. However the ONLY thing different that stands out ATM is the domains being different.
Wondering if anyone ran into a similar experience and could share their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps?
So, I believe that the clients and servers need to be in the same KRB5 domain. This would likely be 123.xyz in your case, rather than using the sub-domains.
K.
Will have to add the realm section etc then. Need the one nfs server shared between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz is the ad dc in this case.
Apologies, I misread I think. Thinking you mean there's only one domain configuration option in the NFS Ganesha config hence why NFS Ganesha only supports a single domain?
/etc/ganesha/ganesha.conf NFSv4 { Lease_Lifetime = 20 ; IdmapConf = "/etc/idmapd.conf" ; DomainName = "b.abc.123" ; } NFS_KRB5 { PrincipalName = "nfs/nfs01.b.abc.123@B.ABC.123" ; KeytabPath = /etc/krb5.keytab ; Active_krb5 = YES ; }
# cat /etc/idmapd.conf [General] Domain = b.abc.123 [Mapping] [Translation] [Static] [UMICH_SCHEMA] LDAP_server = ldap-server.local.domain.edu LDAP_base = dc=local,dc=domain,dc=edu You have new mail in /var/spool/mail/root
The user is the same account on the second subdomain but the trust with the AD DC through which it's served is formed with a separate Linux KDC on that domain.
Perhaps this diagram will come through correctly
sam@abc.123 <-- IPA KDC 1 (a.abc.123) -> AD DC (abc.123) sam@abc.123 <-- IPA KDC 2 (b.abc.123) -----^
The NFS cluster is shared between IPA KDC 1 and 2 .
So in this case, if I wanted this scenario to work, guessing I may have to create a separate NFS cluster and extend the Gluster FS over 3 new additional nodes on domain a.abc.123 to ensure the same user's files and storage is always in sync across domain / kerberos realm.
Reason for all this is that we're testing options for isolating certain software / projects from other domains (keeping them contained within a specific realm). The software can modify the KDC extensively and hence why we prefer it has it's own KDC.
Yes, I think you pretty much need two separate NFS server clusters.
If you only use NFS v4.x, you might be able to have them run on the same nodes by assigning one to a different port, but then the clients need to specify the port to connect to so they get the right KDC.
That Domain parameter in the NFSv4 config block I think only gets used if you don't use idmapd.
I'm not sure how or if idmapd works with multiple domains (Ganesha just passes the domain through, everything is configured using idmapd.conf). But I don't think you need multiple idmap domains.
So I think the issue is just going to come down to the server only has a single Kerberos service principal specified by PrincipleName in the NFS_KRB5 config block. That principal of course belongs to a single Kerberos realm.
Frank
Note, you don't need separate Gluster clusters. There's no requirement that a Ganesha instance run on the same server as one of the bricks, so you can just add a second Ganesha instance pointed at the same Gluster cluster. Daniel
On 2/22/2019 10:45 AM, Daniel Gryniewicz wrote:
On 2/22/19 10:42 AM, Frank Filz wrote:
On 2/21/2019 11:34 AM, Tom wrote:
Sent from my iPhone
On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang@redhat.com> wrote:
On 2/21/19 8:07 AM, TomK wrote: Hey All, Wondering if there would be a cross domain issue if KRB5 is enabled on NFS
Users get a permission denied on clients when accessing the remote mount and while checking I see it's owned by nobody / nobody rather then
The client is on a different DNS domain and ownership comes through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm
Wondering if anyone ran into a similar experience and could share their solution? Do I need to add the realm to the /etc/krb5.conf config on
Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? their user and group. thinking. However the ONLY thing different that stands out ATM is the domains being different. the NFS Ganesha servers perhaps?
So, I believe that the clients and servers need to be in the same KRB5 domain.
This would likely be 123.xyz in your case, rather than using the sub-domains.
K.
Will have to add the realm section etc then. Need the one nfs server shared between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz is the ad dc in this case.
Apologies, I misread I think. Thinking you mean there's only one domain configuration option in the NFS Ganesha config hence why NFS Ganesha only supports a single domain?
/etc/ganesha/ganesha.conf NFSv4 { Lease_Lifetime = 20 ; IdmapConf = "/etc/idmapd.conf" ; DomainName = "b.abc.123" ; } NFS_KRB5 { PrincipalName = "nfs/nfs01.b.abc.123@B.ABC.123" ; KeytabPath = /etc/krb5.keytab ; Active_krb5 = YES ; }
# cat /etc/idmapd.conf [General] Domain = b.abc.123 [Mapping] [Translation] [Static] [UMICH_SCHEMA] LDAP_server = ldap-server.local.domain.edu LDAP_base = dc=local,dc=domain,dc=edu You have new mail in /var/spool/mail/root
The user is the same account on the second subdomain but the trust with the AD DC through which it's served is formed with a separate Linux KDC on that domain.
Perhaps this diagram will come through correctly
sam@abc.123 <-- IPA KDC 1 (a.abc.123) -> AD DC (abc.123) sam@abc.123 <-- IPA KDC 2 (b.abc.123) -----^
The NFS cluster is shared between IPA KDC 1 and 2 .
So in this case, if I wanted this scenario to work, guessing I may have to create a separate NFS cluster and extend the Gluster FS over 3 new additional nodes on domain a.abc.123 to ensure the same user's files and storage is always in sync across domain / kerberos realm.
Reason for all this is that we're testing options for isolating certain software / projects from other domains (keeping them contained within a specific realm). The software can modify the KDC extensively and hence why we prefer it has it's own KDC.
Yes, I think you pretty much need two separate NFS server clusters.
If you only use NFS v4.x, you might be able to have them run on the same nodes by assigning one to a different port, but then the clients need to specify the port to connect to so they get the right KDC.
That Domain parameter in the NFSv4 config block I think only gets used if you don't use idmapd.
I'm not sure how or if idmapd works with multiple domains (Ganesha just passes the domain through, everything is configured using idmapd.conf). But I don't think you need multiple idmap domains.
So I think the issue is just going to come down to the server only has a single Kerberos service principal specified by PrincipleName in the NFS_KRB5 config block. That principal of course belongs to a single Kerberos realm.
Frank
Note, you don't need separate Gluster clusters. There's no requirement that a Ganesha instance run on the same server as one of the bricks, so you can just add a second Ganesha instance pointed at the same Gluster cluster.
Daniel _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Manged to get this working with a single implementation of NFS Ganesha and . Linux KDC on domain a.abc.xyz and Linux KDC on domain b.abc.xyz have a trust to the _same_ upstream AD DC. This means that sam@abc.xyz has the same SID on both because it's the same user from the same AD DC presented via a separate trust to each of the Linux KDC's. Because the SID is the same, SSSD generates the same UID / GID. Because the same UID / GID is presented to hosts under the two different domains, the same NFS folder is accessible on NFS Ganesha mounted shares despite the configured domain being that of a.abc.xyz on the NFS Ganesha Servers. And so this is correct because that means the same user, and only that user, has access to their folder. For users defined individually on the two Linux KDC's (ie not in AD through which these two KDC's have a trust with) then separate folders will exist on NFS Ganesha and neither user would be able to see the other's files since each would exist on a different domain ( bob@a.abc.xyz UID / GID != bob@b.abc.xyz UID / GID ) and be given different UID / GID numbers. This effectively resolved the issue here. -- Cheers, Tom K. ------------------------------------------------------------------------------------- Living on earth is expensive, but it includes a free trip around the sun.
-----Original Message----- From: TomK [mailto:tomkcpr@mdevsys.com] Sent: Monday, February 25, 2019 5:48 PM To: dang@redhat.com; Frank Filz <ffilzlnx@mindspring.com>; 'Tom' <tk@mdevsys.com> Cc: support@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Support] Re: KRB5, nfs-ganesha and directories appearing with ownership as nobody/nobody on client side.
On 2/22/2019 10:45 AM, Daniel Gryniewicz wrote:
On 2/22/19 10:42 AM, Frank Filz wrote:
On 2/21/2019 11:34 AM, Tom wrote:
Sent from my iPhone
On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang@redhat.com> wrote:
> On 2/21/19 8:07 AM, TomK wrote: > Hey All, > Wondering if there would be a cross domain issue if KRB5 is > enabled on NFS
> Users get a permission denied on clients when accessing the > remote mount and while checking I see it's owned by nobody / nobody rather then
Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? their user and group.
> The client is on a different DNS domain and ownership comes > through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm thinking. However the ONLY thing different that stands out ATM is the domains being different. > Wondering if anyone ran into a similar experience and could share > their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps?
So, I believe that the clients and servers need to be in the same KRB5 domain. This would likely be 123.xyz in your case, rather than using the sub-domains.
K.
Will have to add the realm section etc then. Need the one nfs server shared between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz is the ad dc in this case.
Apologies, I misread I think. Thinking you mean there's only one domain configuration option in the NFS Ganesha config hence why NFS Ganesha only supports a single domain?
/etc/ganesha/ganesha.conf NFSv4 { Lease_Lifetime = 20 ; IdmapConf = "/etc/idmapd.conf" ; DomainName = "b.abc.123" ; } NFS_KRB5 { PrincipalName = "nfs/nfs01.b.abc.123@B.ABC.123" ; KeytabPath = /etc/krb5.keytab ; Active_krb5 = YES ; }
# cat /etc/idmapd.conf [General] Domain = b.abc.123 [Mapping] [Translation] [Static] [UMICH_SCHEMA] LDAP_server = ldap-server.local.domain.edu LDAP_base = dc=local,dc=domain,dc=edu You have new mail in /var/spool/mail/root
The user is the same account on the second subdomain but the trust with the AD DC through which it's served is formed with a separate Linux KDC on that domain.
Perhaps this diagram will come through correctly
sam@abc.123 <-- IPA KDC 1 (a.abc.123) -> AD DC (abc.123) sam@abc.123 <-- IPA KDC 2 (b.abc.123) -----^
The NFS cluster is shared between IPA KDC 1 and 2 .
So in this case, if I wanted this scenario to work, guessing I may have to create a separate NFS cluster and extend the Gluster FS over 3 new additional nodes on domain a.abc.123 to ensure the same user's files and storage is always in sync across domain / kerberos realm.
Reason for all this is that we're testing options for isolating certain software / projects from other domains (keeping them contained within a specific realm). The software can modify the KDC extensively and hence why we prefer it has it's own KDC.
Yes, I think you pretty much need two separate NFS server clusters.
If you only use NFS v4.x, you might be able to have them run on the same nodes by assigning one to a different port, but then the clients need to specify the port to connect to so they get the right KDC.
That Domain parameter in the NFSv4 config block I think only gets used if you don't use idmapd.
I'm not sure how or if idmapd works with multiple domains (Ganesha just passes the domain through, everything is configured using idmapd.conf). But I don't think you need multiple idmap domains.
So I think the issue is just going to come down to the server only has a single Kerberos service principal specified by PrincipleName in the NFS_KRB5 config block. That principal of course belongs to a single Kerberos realm.
Frank
Note, you don't need separate Gluster clusters. There's no requirement that a Ganesha instance run on the same server as one of the bricks, so you can just add a second Ganesha instance pointed at the same Gluster cluster.
Daniel _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Manged to get this working with a single implementation of NFS Ganesha and .
Linux KDC on domain a.abc.xyz and Linux KDC on domain b.abc.xyz have a trust to the _same_ upstream AD DC. This means that sam@abc.xyz has the same SID on both because it's the same user from the same AD DC presented via a separate trust to each of the Linux KDC's.
Because the SID is the same, SSSD generates the same UID / GID.
Because the same UID / GID is presented to hosts under the two different domains, the same NFS folder is accessible on NFS Ganesha mounted shares despite the configured domain being that of a.abc.xyz on the NFS Ganesha Servers. And so this is correct because that means the same user, and only that user, has access to their folder.
For users defined individually on the two Linux KDC's (ie not in AD through which these two KDC's have a trust with) then separate folders will exist on NFS Ganesha and neither user would be able to see the other's files since each would exist on a different domain ( bob@a.abc.xyz UID / GID != bob@b.abc.xyz UID / GID ) and be given different UID / GID numbers.
This effectively resolved the issue here.
Great. If you want to write something up that would help others, I would be happy to post it to the wiki so others might benefit. Frank
On 2/26/2019 12:13 PM, Frank Filz wrote:
-----Original Message----- From: TomK [mailto:tomkcpr@mdevsys.com] Sent: Monday, February 25, 2019 5:48 PM To: dang@redhat.com; Frank Filz <ffilzlnx@mindspring.com>; 'Tom' <tk@mdevsys.com> Cc: support@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Support] Re: KRB5, nfs-ganesha and directories appearing with ownership as nobody/nobody on client side.
On 2/22/2019 10:45 AM, Daniel Gryniewicz wrote:
On 2/22/19 10:42 AM, Frank Filz wrote:
On 2/21/2019 11:34 AM, Tom wrote:
Sent from my iPhone
> On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang@redhat.com> > wrote: > >> On 2/21/19 8:07 AM, TomK wrote: >> Hey All, >> Wondering if there would be a cross domain issue if KRB5 is >> enabled on NFS
>> Users get a permission denied on clients when accessing the >> remote mount and while checking I see it's owned by nobody / nobody rather then
Ganesha and thereby causing an issue if the NFS Ganesha servers are on say, domain aaa.123.xyz while the clients are on domain bbb.123.xyz? their user and group.
>> The client is on a different DNS domain and ownership comes >> through with nobody/nobody. Hence clients get a permission denied accessing the folder I'm thinking. However the ONLY thing different that stands out ATM is the domains being different. >> Wondering if anyone ran into a similar experience and could share >> their solution? Do I need to add the realm to the /etc/krb5.conf config on the NFS Ganesha servers perhaps? > > So, I believe that the clients and servers need to be in the same > KRB5 domain. This would likely be 123.xyz in your case, rather than using the sub-domains.
K.
Will have to add the realm section etc then. Need the one nfs server shared between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz is the ad dc in this case.
Apologies, I misread I think. Thinking you mean there's only one domain configuration option in the NFS Ganesha config hence why NFS Ganesha only supports a single domain?
/etc/ganesha/ganesha.conf NFSv4 { Lease_Lifetime = 20 ; IdmapConf = "/etc/idmapd.conf" ; DomainName = "b.abc.123" ; } NFS_KRB5 { PrincipalName = "nfs/nfs01.b.abc.123@B.ABC.123" ; KeytabPath = /etc/krb5.keytab ; Active_krb5 = YES ; }
# cat /etc/idmapd.conf [General] Domain = b.abc.123 [Mapping] [Translation] [Static] [UMICH_SCHEMA] LDAP_server = ldap-server.local.domain.edu LDAP_base = dc=local,dc=domain,dc=edu You have new mail in /var/spool/mail/root
The user is the same account on the second subdomain but the trust with the AD DC through which it's served is formed with a separate Linux KDC on that domain.
Perhaps this diagram will come through correctly
sam@abc.123 <-- IPA KDC 1 (a.abc.123) -> AD DC (abc.123) sam@abc.123 <-- IPA KDC 2 (b.abc.123) -----^
The NFS cluster is shared between IPA KDC 1 and 2 .
So in this case, if I wanted this scenario to work, guessing I may have to create a separate NFS cluster and extend the Gluster FS over 3 new additional nodes on domain a.abc.123 to ensure the same user's files and storage is always in sync across domain / kerberos realm.
Reason for all this is that we're testing options for isolating certain software / projects from other domains (keeping them contained within a specific realm). The software can modify the KDC extensively and hence why we prefer it has it's own KDC.
Yes, I think you pretty much need two separate NFS server clusters.
If you only use NFS v4.x, you might be able to have them run on the same nodes by assigning one to a different port, but then the clients need to specify the port to connect to so they get the right KDC.
That Domain parameter in the NFSv4 config block I think only gets used if you don't use idmapd.
I'm not sure how or if idmapd works with multiple domains (Ganesha just passes the domain through, everything is configured using idmapd.conf). But I don't think you need multiple idmap domains.
So I think the issue is just going to come down to the server only has a single Kerberos service principal specified by PrincipleName in the NFS_KRB5 config block. That principal of course belongs to a single Kerberos realm.
Frank
Note, you don't need separate Gluster clusters. There's no requirement that a Ganesha instance run on the same server as one of the bricks, so you can just add a second Ganesha instance pointed at the same Gluster cluster.
Daniel _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
Manged to get this working with a single implementation of NFS Ganesha and .
Linux KDC on domain a.abc.xyz and Linux KDC on domain b.abc.xyz have a trust to the _same_ upstream AD DC. This means that sam@abc.xyz has the same SID on both because it's the same user from the same AD DC presented via a separate trust to each of the Linux KDC's.
Because the SID is the same, SSSD generates the same UID / GID.
Because the same UID / GID is presented to hosts under the two different domains, the same NFS folder is accessible on NFS Ganesha mounted shares despite the configured domain being that of a.abc.xyz on the NFS Ganesha Servers. And so this is correct because that means the same user, and only that user, has access to their folder.
For users defined individually on the two Linux KDC's (ie not in AD through which these two KDC's have a trust with) then separate folders will exist on NFS Ganesha and neither user would be able to see the other's files since each would exist on a different domain ( bob@a.abc.xyz UID / GID != bob@b.abc.xyz UID / GID ) and be given different UID / GID numbers.
This effectively resolved the issue here.
Great. If you want to write something up that would help others, I would be happy to post it to the wiki so others might benefit.
Frank
More then happy to do so and make a specific page on a Ganesha Wiki. Would it be this Wiki: https://github.com/nfs-ganesha/nfs-ganesha/wiki ? I should note that posts I've put together earlier are available at these links and are available to anyone: Guest (Ganesha): https://tinyurl.com/y2n4jxe3 Ganesha: https://tinyurl.com/y2ag7u6q FreeIPA: https://tinyurl.com/y42mmtuk However though all this does work, I'm not sure the issue is 'fully' solved with files and folders showing up with nobody/nobody from the secondary domain. Though I got a temporary impression it may have been. Interestingly, functionally and from the UID / GID standpoint, all of it works fine and is what it should be. Take a look below and see what happens once I change the owner of a single file on the mount from a host on the new domain: 11940739822224053652 -rw-r--r--. 1 nobody nobody 22 Feb 24 16:28 hello.txt 9337343577229627320 -rw-------. 1 nobody nobody 9874 Feb 27 00:04 .bash_history 12094630883843656556 -rw-rw-r--. 1 nobody nobody 6 Feb 27 22:07 test-it.out 11438151930727967183 drwx------. 8 nobody nobody 4096 Feb 27 22:07 . sam@abc.123@cm-r01nn02:~] :) $ chown sam@abc.123:sam@abc.123 test-it.out sam@abc.123@cm-r01nn02:~] :) $ ls -altri total 754252 11940739822224053652 -rw-r--r--. 1 sam@abc.123 sam@abc.123 22 Feb 24 16:28 hello.txt 9337343577229627320 -rw-------. 1 sam@abc.123 sam@abc.123 9874 Feb 27 00:04 .bash_history 12094630883843656556 -rw-rw-r--. 1 sam@abc.123 sam@abc.123 6 Feb 27 22:07 test-it.out 11438151930727967183 drwx------. 8 nobody nobody 4096 Feb 27 22:07 . sam@abc.123@cm-r01nn01:~] :) $ pwd /n/abc.123/tom sam@abc.123@cm-r01nn01:~] :) $ hostname cm-r01nn01.b.abc.123 sam@abc.123@cm-r01nn01:~] :) $ Notice how the display in the putty window relabels all the files with sam@abc.123 even though I only ran chmod on a single file? chmod I think did absolutely nothing since ownership has not changed when looking at this from the original domain ( a.abc.123 ) however it's odd how the rest of the files are displaying properly now in the same terminal. So I'm wondering if this issue is really on the host or is more with something else not displaying the ownership properly. I suppose I can understand this further by stracing chmod and seeing exactly what it touches when 'apparently' making that change. The ID of the user on both domains is below. Files on that NFS space are owned by this UID / GID: uid=155601104(sam@abc.123) gid=155601104(sam@abc.123) Don't have too much time to take a look at this this moment but will try to do so more over the next week. Packages I'm using: [root@nfs01 ~]# rpm -aq|grep -Ei ganesha nfs-ganesha-2.6.1-1.el7.x86_64 nfs-ganesha-vfs-2.6.1-1.el7.x86_64 nfs-ganesha-gluster-2.6.1-1.el7.x86_64 nfs-ganesha-xfs-2.6.1-1.el7.x86_64 nfs-ganesha-mount-9P-2.6.1-1.el7.x86_64 nfs-ganesha-utils-2.6.1-1.el7.x86_64 nfs-ganesha-proxy-2.6.1-1.el7.x86_64 [root@nfs01 ~]#
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org
-- Cheers, Tom K. ------------------------------------------------------------------------------------- Living on earth is expensive, but it includes a free trip around the sun.
participants (4)
-
Daniel Gryniewicz -
Frank Filz -
Tom -
TomK