[NFS-Ganesha-Support] Re: KRB5, nfs-ganesha and directories appearing with ownership as nobody/nobody on client side.

> -----Original Message----- > From: TomK [mailto:tomkcpr@mdevsys.com] > Sent: Monday, February 25, 2019 5:48 PM > To: dang(a)redhat.com; Frank Filz <ffilzlnx(a)mindspring.com>; 'Tom' > <tk(a)mdevsys.com&gt; > Cc: support(a)lists.nfs-ganesha.org > Subject: [NFS-Ganesha-Support] Re: KRB5, nfs-ganesha and directories > appearing with ownership as nobody/nobody on client side. > > On 2/22/2019 10:45 AM, Daniel Gryniewicz wrote: >> On 2/22/19 10:42 AM, Frank Filz wrote: >>> On 2/21/2019 11:34 AM, Tom wrote: >>>>> >>>>> >>>>> Sent from my iPhone >>>>> >>>>>> On Feb 21, 2019, at 8:43 AM, Daniel Gryniewicz <dang(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> On 2/21/19 8:07 AM, TomK wrote: >>>>>>> Hey All, >>>>>>> Wondering if there would be a cross domain issue if KRB5 is >>>>>>> enabled on NFS >>>> Ganesha and thereby causing an issue if the NFS Ganesha servers are >>>> on say, domain aaa.123.xyz while the clients are on domain >>>> bbb.123.xyz? >>>>>>> Users get a permission denied on clients when accessing the >>>>>>> remote mount >>>> and while checking I see it's owned by nobody / nobody rather then >>>> their user and group. >>>>>>> The client is on a different DNS domain and ownership comes >>>>>>> through with >>>> nobody/nobody. Hence clients get a permission denied accessing the >>>> folder I'm thinking. However the ONLY thing different that stands >>>> out ATM is the domains being different. >>>>>>> Wondering if anyone ran into a similar experience and could share >>>>>>> their >>>> solution? Do I need to add the realm to the /etc/krb5.conf config >>>> on the NFS Ganesha servers perhaps? >>>>>> >>>>>> So, I believe that the clients and servers need to be in the same >>>>>> KRB5 domain. >>>> This would likely be 123.xyz in your case, rather than using the >>>> sub-domains. >>>>> >>>>> K. >>>>> >>>>> Will have to add the realm section etc then. Need the one nfs >>>>> server shared >>>> between two kdc ‘s . I have a separate kdc per subdomain. ABC.xyz >>>> is the ad dc in this case. >>>> >>>> Apologies, I misread I think. Thinking you mean there's only one >>>> domain configuration option in the NFS Ganesha config hence why NFS >>>> Ganesha only supports a single domain? >>>> >>>> /etc/ganesha/ganesha.conf >>>> NFSv4 { >>>> Lease_Lifetime = 20 ; >>>> IdmapConf = "/etc/idmapd.conf" ; >>>> DomainName = "b.abc.123" ; >>>> } >>>> NFS_KRB5 { >>>> PrincipalName = &quot;nfs/nfs01.b.abc.123(a)B.ABC.123&quot; ; >>>> KeytabPath = /etc/krb5.keytab ; >>>> Active_krb5 = YES ; >>>> } >>>> >>>> >>>> # cat /etc/idmapd.conf >>>> [General] >>>> Domain = b.abc.123 >>>> [Mapping] >>>> [Translation] >>>> [Static] >>>> [UMICH_SCHEMA] >>>> LDAP_server = ldap-server.local.domain.edu LDAP_base = >>>> dc=local,dc=domain,dc=edu You have new mail in /var/spool/mail/root >>>> >>>> The user is the same account on the second subdomain but the trust >>>> with the AD DC through which it's served is formed with a separate >>>> Linux KDC on that domain. >>>> >>>> Perhaps this diagram will come through correctly >>>> >>>> sam(a)abc.123 <-- IPA KDC 1 (a.abc.123) -> AD DC (abc.123) >>>> sam(a)abc.123 <-- IPA KDC 2 (b.abc.123) -----^ >>>> >>>> The NFS cluster is shared between IPA KDC 1 and 2 . >>>> >>>> So in this case, if I wanted this scenario to work, guessing I may >>>> have to create a separate NFS cluster and extend the Gluster FS over >>>> 3 new additional nodes on domain a.abc.123 to ensure the same user's >>>> files and storage is always in sync across domain / kerberos realm. >>>> >>>> Reason for all this is that we're testing options for isolating >>>> certain software / projects from other domains (keeping them >>>> contained within a specific realm). >>>> The software can modify the KDC extensively and hence why we prefer >>>> it has it's own KDC. >>> >>> Yes, I think you pretty much need two separate NFS server clusters. >>> >>> If you only use NFS v4.x, you might be able to have them run on the >>> same nodes by assigning one to a different port, but then the clients >>> need to specify the port to connect to so they get the right KDC. >>> >>> That Domain parameter in the NFSv4 config block I think only gets >>> used if you don't use idmapd. >>> >>> I'm not sure how or if idmapd works with multiple domains (Ganesha >>> just passes the domain through, everything is configured using >>> idmapd.conf). But I don't think you need multiple idmap domains. >>> >>> So I think the issue is just going to come down to the server only >>> has a single Kerberos service principal specified by PrincipleName in >>> the >>> NFS_KRB5 config block. That principal of course belongs to a single >>> Kerberos realm. >>> >>> Frank >>> >> >> Note, you don't need separate Gluster clusters. There's no >> requirement that a Ganesha instance run on the same server as one of >> the bricks, so you can just add a second Ganesha instance pointed at >> the same Gluster cluster. >> >> Daniel >> _______________________________________________ >> Support mailing list -- support(a)lists.nfs-ganesha.org To unsubscribe >> send an email to support-leave(a)lists.nfs-ganesha.org > > Manged to get this working with a single implementation of NFS Ganesha and . > > Linux KDC on domain a.abc.xyz and Linux KDC on domain b.abc.xyz have a trust > to the _same_ upstream AD DC. This means that sam(a)abc.xyz has the same SID > on both because it's the same user from the same AD DC presented via a > separate trust to each of the Linux KDC's. > > Because the SID is the same, SSSD generates the same UID / GID. > > Because the same UID / GID is presented to hosts under the two different > domains, the same NFS folder is accessible on NFS Ganesha mounted shares > despite the configured domain being that of a.abc.xyz on the NFS Ganesha > Servers. And so this is correct because that means the same user, and only that > user, has access to their folder. > > For users defined individually on the two Linux KDC's (ie not in AD through which > these two KDC's have a trust with) then separate folders will exist on NFS > Ganesha and neither user would be able to see the other's files since each would > exist on a different domain ( bob(a)a.abc.xyz UID / GID != bob(a)b.abc.xyz UID / > GID ) and be given different UID / GID numbers. > > This effectively resolved the issue here. Great. If you want to write something up that would help others, I would be happy to post it to the wiki so others might benefit. Frank

_______________________________________________ Support mailing list -- support(a)lists.nfs-ganesha.org To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org