September 2020 - Devel - NFS Ganesha List Archives

Change in ...nfs-ganesha[next]: Allow EXPORT pseudo path to be changed during export update
by Frank Filz (GerritHub) 20 Feb '24

20 Feb '24

Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490334 ) Change subject: Allow EXPORT pseudo path to be changed during export update ...................................................................... Allow EXPORT pseudo path to be changed during export update This also fully allows adding or removing NFSv4 support from an export since we can now handle the PseudoFS swizzing that occurs. Note that an explicit PseudoFS export may be removed or added, though you can not change it from export_id 0 because we currently don't allow changing the export_id. Note that this patch doesn't handle DBUS add or remove export though that is an option to improve. I may add them to this patch (it wouldn't be that hard) but I want to get this reviewed as is right now. There are implications to a client of changing the PseudoFS. I have tested moving an export in the PseudoFS with a client mounted. The client will be able to continue accessing the export, though it may see an ESTALE error if it navigates out of the export. The current working directory will go bad and the pwd comment will fail indicating a disconnected mount. I have also seen referencing .. from the root of the export wrapping around back to the root (I believe this is how disconnected mounts are set up). FSAL_PSEUDO lookups and create handles (PUTFH or any use of an NFSv3 handle where the inode isn't cached) which fail during an export update are instead turned into ERR_FSAL_DELAY which turns into NFS4ERR_DELAY or NFS3ERR_JUKEBOX to force the client to retry under the completed update. Change-Id: I507dc17a651936936de82303ff1291677ce136be Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> --- M src/FSAL/FSAL_PSEUDO/handle.c M src/MainNFSD/libganesha_nfsd.ver M src/Protocols/NFS/nfs4_pseudo.c M src/include/export_mgr.h M src/include/nfs_proto_functions.h M src/support/export_mgr.c M src/support/exports.c 7 files changed, 560 insertions(+), 203 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/34/490334/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490334 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I507dc17a651936936de82303ff1291677ce136be Gerrit-Change-Number: 490334 Gerrit-PatchSet: 1 Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com> Gerrit-MessageType: newchange

2 2

Monitoring in Ganesha?
by Bjorn Leffler 22 Nov '21

22 Nov '21

Apart from the counters that you can access through dbus, is there any other monitoring built into Ganesha? I'm thinking of adding it with this higher level plan: - Exporting metrics from Ganesha to Prometheus. - Aggregate data in Prometheus. - Display monitoring consoles and graphs with Grafana. - Package up Prometheus, Grafana and the preconfigured rules/dashboards as a Docker image. - This makes it straightforward to deploy monitoring alongside a Gansha binary. My rough coding plan for the code is to: - Add a USE_MONITORING directive to the CMakeLists.txt files. - Add a build dependency to the Prometheus C client <https://github.com/digitalocean/prometheus-client-c>. - Create a src/monitoring directory for the new source files and templates. - Increment counters and timers throughout the code. - Use histograms to compute latency percentiles, heatmaps, etc. Is this a good idea? Any objections or suggestions? Thanks, Bjorn

3 6

Change in ...nfs-ganesha[next]: MDCACHE - Add MDCACHE {} config block
by Daniel Gryniewicz (GerritHub) 05 Nov '20

05 Nov '20

Daniel Gryniewicz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/454929 Change subject: MDCACHE - Add MDCACHE {} config block ...................................................................... MDCACHE - Add MDCACHE {} config block Add a config block name MDCACHE that is a copy of CACHEINODE. Both can be configured, but MDCACHE will override CACHEINODE. This allows us to deprecate CACHEINODE. Change-Id: I49012723132ae6105b904a60d1a96bb2bf78d51b Signed-off-by: Daniel Gryniewicz <dang(a)fprintf.net> --- M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_read_conf.c M src/config_samples/ceph.conf M src/config_samples/config.txt M src/config_samples/ganesha.conf.example M src/doc/man/ganesha-cache-config.rst M src/doc/man/ganesha-config.rst 6 files changed, 31 insertions(+), 7 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/29/454929/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/454929 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I49012723132ae6105b904a60d1a96bb2bf78d51b Gerrit-Change-Number: 454929 Gerrit-PatchSet: 1 Gerrit-Owner: Daniel Gryniewicz <dang(a)redhat.com> Gerrit-MessageType: newchange

3 2

Issues seen with krb5i and krb5p mounts
by Trishali Nayar 21 Oct '20

21 Oct '20

Hi all, There are a few clients eg- Ubuntu 18.04.3 (4.15.0-55-generic) and RH7.8 (3.10.0-1127.el7.x86_64) for which we have observed... simple command like 'dd' either hangs or returns EIO. This is happening only on krb5i and krb5p mounts. It seems to happen for file sizes eg- 100MB and larger mostly. But sometimes even a 30 MB file sees failures. A client eg- RH7.6 (3.10.0-957.el7.x86_64) does not seem to hit this issue...so might be with more recent kernels? We fixed the issue with check-in https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490802 The idea was to let clients know that Ganesha denied the request VS just dropping the request. This fix did seem to help and hangs/errors stopped completely... but for larger file sizes eg- 1000MB we started seeing "Permission Denied" errors. This was different than the EIO errors seen earlier. Reason could be we are now sending an "AUTH DENIED" error so clients translate it to this new error. We tried to add more logging into Ganesha and observe that these particular clients seem to send a lot of requests together. When we process same, the sequence no. is pretty much out or order and we drop the requests outside the sequence window, as per the RFC 2203 Section 7.2.1. The sequence window that we have is 32. Testing these clients with kNFS does not hit the issue...The kNFS sequence window seems to be larger and is 128. So, tried to increase the sequence window as well to 128 for ganesha. That does not seem to help fix the issue. We also have below additional 'seqmask' check and many of the requests went into that category as well and were dropped. "libntirpc/src/svc_auth_gss.c": } else if (offset >= gd->win || (gd->seqmask & (1 << offset))) { *no_dispatch = true; goto gd_free; } Also observed that now these clients sent many requests above the 128 window...which we would again drop. Wondering what is the proper way to fix this and any idea on what these clients are doing different. Thanks and regards, Trishali. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Trishali Nayar IBM Systems ETZ, Pune. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

6 18

Seeking assistance with Ganesha PROXY FSAL / libntirpc issues
by Jo Seaton 30 Sep '20

30 Sep '20

Hi all, I've been working on re-adding GSS/Kerberos authentication support to the PROXY (V4) FSAL, with a mind to eventually also adding Kerberos delegation support for completeness. I've been having some issues I was hoping to get some feedback on. The first issue I've been having is to do with the GSS code in libntirpc. I'm calling authgss_ncreate_default with a valid CLIENT * and service name, and hopefully a reasonable struct rpc_gss_sec. This fails, because the call to gss_verify_mic inside authgss_refresh fails. This appears to be because authgss_verify fills a relevant buffer (gd->gc_wire_verf) with <empty>, which originally comes from cc_verf in the clnt code (via the start of authgss_validate). Specifically I'm looking at clnt_req's cc_verf which gets used for AUTH_VALIDATE in clnt_generic.c, and always seems to have the same "_null_auth" value - which seems surprising to me! If anyone can give me some insight into what exactly cc_verf is supposed to contain that might help me fix it. Working around it by ignoring the result of gss_verify_mic does seem to work OK. The second issue is to do with the structure of the PROXY FSAL. It appears that it largely handles requests "manually", calling the relevant xdr_* functions, and reading/writing to sockets itself. The GSS auth code on the other hand, seems to require use of CLIENT *, which in my understanding means handing responsibility for the socket to that CLIENT *. These two approaches appear incompatible to me. I've made some reasonable progress rewriting PROXY with clnt_req_* functions, similar to nfs_rpc_callback.c, but if anyone has any feedback on a) why the original approach (ffilz suggests those functions didn't used to be threadsafe?) b) the most sensible thing to do now, it would be very much appreciated. Anyway any feedback is very welcome, I'm very new to both Ganesha and GSS/Kerberos. Many thanks, Jo

4 8

Readdirplus entries without attributes?
by Bjorn Leffler 30 Sep '20

30 Sep '20

How does Ganesha (especially MDCACHE) handle readdir responses when some or all files have no attributes? Is this supported and supposed to work? I'm testing proxy configurations against a NetApp filer. NetApp filers don't return attributes in the readdir response for .snapshot directories. Thanks, Bjorn

3 8

Change in ...nfs-ganesha[next]: grace: avoid add duplicate client entries from backend
by sepia-liu (GerritHub) 30 Sep '20

30 Sep '20

sepia-liu has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503292 ) Change subject: grace: avoid add duplicate client entries from backend ...................................................................... grace: avoid add duplicate client entries from backend If ganesha is already in grace period, at the point, it enter grace by admin_dbus_grace() again, it is likely that add duplicate client entries from same backend object. The result is that clid_count never equal to reclaim_completes. Change-Id: I565f807fcd5e806717544a2a8d32f0249463afa1 Signed-off-by: sepia-liu <liuwei_coder(a)163.com> --- M src/SAL/nfs4_recovery.c 1 file changed, 20 insertions(+), 1 deletion(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/92/503292/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503292 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I565f807fcd5e806717544a2a8d32f0249463afa1 Gerrit-Change-Number: 503292 Gerrit-PatchSet: 1 Gerrit-Owner: sepia-liu <liuwei_coder(a)163.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: FSAL_CEPH: fix create handle return EINVAL
by freeze (GerritHub) 30 Sep '20

30 Sep '20

freeze has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503258 ) Change subject: FSAL_CEPH: fix create handle return EINVAL ...................................................................... FSAL_CEPH: fix create handle return EINVAL - correct the compare with key and len on the right data type. Change-Id: I2a1d8c0e5b1d74a5eb51cd894a7825ba59ce573c Signed-off-by: Vicente Cheng <vicente_cheng(a)bigtera.com> --- M src/FSAL/FSAL_CEPH/export.c 1 file changed, 5 insertions(+), 4 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/58/503258/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503258 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I2a1d8c0e5b1d74a5eb51cd894a7825ba59ce573c Gerrit-Change-Number: 503258 Gerrit-PatchSet: 1 Gerrit-Owner: freeze <freeze.bilsted(a)gmail.com> Gerrit-MessageType: newchange

1 0

Re: SELinux and GlusterFS
by jiffin tony Thottan 28 Sep '20

28 Sep '20

Hey, The selinux feature supported in glusterfs, but for fuse clients there is some patch missing in kernel side AFAIR. The feature works with nfs4 via nfs-ganesha. Currently I am not actively working on those projects. Adding Arjun in loop who worked for the support in nfs-ganesha. Also ccing dev list of both projects. -- Jiffin On Fri, 25 Sep 2020 at 3:07 AM, Karl MacMillan <karl(a)bigbadwolfsecurity.com> wrote: > Hey, > > I saw your Fosdem and other presentations about GlusterFS and SELinux - > last I saw was about missing 3.10 merge. I'm looking for a distributed > filesystem with SELinux support and was wondering what the current status > is. Sorry to bother you with email - I just couldn't find anything > definitive about where things are at the moment. > > If GlusterFS isn't ready, are you aware of another option? I guess maybe > NFSv4? But that's not exactly the same kind of thing. > > Thanks - Karl > > > -- Jiffin Tony Thottan Ph:+918129412217

1 0

Change in ...nfs-ganesha[next]: grace: optimize takeip process to end grace early
by sepia-liu (GerritHub) 27 Sep '20

27 Sep '20

sepia-liu has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503054 ) Change subject: grace: optimize takeip process to end grace early ...................................................................... grace: optimize takeip process to end grace early For NFS4.1+, the agreement allows end grace in advence, clid_count and reclaim_completes are only effective during grace period, They should not be cumulative value, in this way, the last reclaim result will affact the next takeip process. If reclaim_completes not equal clid_count in one time(some clients nerver come back), it will lead to the following grace not being able to end early. Change-Id: I9bc5f95c3bf7a889f62978b7a4f879f11d6b43ed Signed-off-by: sepia-liu <liuwei_coder(a)163.com> --- M src/SAL/nfs4_recovery.c 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/54/503054/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503054 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I9bc5f95c3bf7a889f62978b7a4f879f11d6b43ed Gerrit-Change-Number: 503054 Gerrit-PatchSet: 1 Gerrit-Owner: sepia-liu <liuwei_coder(a)163.com> Gerrit-MessageType: newchange

1 0