On Fri, Aug 28, 2020, 16:24 Frank Filz <ffilzlnx@mindspring.com> wrote:

...

Back in April I wrote:

On Thu, 9 Apr 2020, Pfaff, Todd wrote:

...

I'm having stability problems with nfs-ganesha 2.8 and PROXY FSAL on CentOS 7.

and this has been ongoing for me since then. Today I finally stumbled onto something that may have solved at least some of the problems that I've been having with nfs-ganesha. I also posted the following as a follow up to the nfs- ganesha github issue: frequent failures with PROXY FSAL (#580).

...

I've discovered something that may be relevant to this issue.

- My nfs-ganesha server is serving several EXPORTs.

- Two of these EXPORTs refer to identically named PATHs on two different back-end servers.

Are the exports different though? Meaning both are exporting /mnt or whatever, but one is doing so as /foo and the other as /bar? Can you share the example here?

...

- I've now changed these so that the back-end server paths are different.

- I've tested this multiple times with multiple VMs and it appears that this change may have eliminated all traces of the problems that I've been describing here.

Does this make sense? Is it known that nfs-ganesha can not have multiple exports with the same value for the Path?

I'm not that familiar with FSAL_PROXY, and I'm not sure how it handles multiple exports. That is actually an area that needs more testing and more resolution.

Do you use

NFS_CORE_PARAM { mount_path_pseudo = true; }

You may be running into issues if path is the same with NFS v3 clients.

If that isn't the issue, we do need to make the proxy support more robust because certainly we should allow multiple back end servers exporting the same path.

Do the different backend servers also have different FSINFO / parameters? (I now forget how clearly or not the two proxy codepaths ensure that they correctly have per-export state/config versus per FSAL config for things like write size, attributes, and so on). The main things that show up as being path dependent are LOOKUP by path (e.g. for initial mounting) and then READDIR caching in the MDCACHE (IIRC). Other things are FH dependent, which I'd be curious if knfsd generates FH3/4 values as consistent with a path (so if both have /mnt/dir as a path on two separate servers, would that produce the same FH3/FH4 handle in response? All of the handle to key stuff assumes handles would be unique).

Frank _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org

Here are the relevant parts of my ganesha.conf related to the two EXPORTs with same Path. This is the state before, when booting guests was consistently failing. NFS_CORE_PARAM { mount_path_pseudo = true; } EXPORT { Export_Id = 10101; Path = /pool/vm-images; Pseudo = /export/ss1/vm-images; #Tag = ss1; SecType = sys; Access_Type = none; Squash = All_Squash; FSAL { Name = PROXY_V3; Srv_Addr=10.168.168.1; } CLIENT { Clients = @vh; Access_Type = RW; Squash = No_Root_Squash; } } EXPORT { Export_Id = 10102; Path = /pool/vm-images; Pseudo = /export/ss2/vm-images; #Tag = ss2; SecType = sys; Access_Type = none; Squash = All_Squash; FSAL { Name = PROXY_V3; Srv_Addr=10.168.168.2; } CLIENT { Clients = @vh; Access_Type = RW; Squash = No_Root_Squash; } } This is the the state after I changed the second EXPORT Path to differ from the first. NFS_CORE_PARAM { mount_path_pseudo = true; } EXPORT { Export_Id = 10101; Path = /pool/vm-images; Pseudo = /export/ss1/vm-images; #Tag = ss1; SecType = sys; Access_Type = none; Squash = All_Squash; FSAL { Name = PROXY_V3; Srv_Addr=10.168.168.1; } CLIENT { Clients = @vh; Access_Type = RW; Squash = No_Root_Squash; } } EXPORT { Export_Id = 10102; Path = /pool/vm-images2; Pseudo = /export/ss2/vm-images; #Tag = ss2; SecType = sys; Access_Type = none; Squash = All_Squash; FSAL { Name = PROXY_V3; Srv_Addr=10.168.168.2; } CLIENT { Clients = @vh; Access_Type = RW; Squash = No_Root_Squash; } }