February 2021 - Devel - NFS Ganesha List Archives

Change in ...nfs-ganesha[next]: Allow EXPORT pseudo path to be changed during export update
by Frank Filz (GerritHub) 20 Feb '24

20 Feb '24

Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490334 ) Change subject: Allow EXPORT pseudo path to be changed during export update ...................................................................... Allow EXPORT pseudo path to be changed during export update This also fully allows adding or removing NFSv4 support from an export since we can now handle the PseudoFS swizzing that occurs. Note that an explicit PseudoFS export may be removed or added, though you can not change it from export_id 0 because we currently don't allow changing the export_id. Note that this patch doesn't handle DBUS add or remove export though that is an option to improve. I may add them to this patch (it wouldn't be that hard) but I want to get this reviewed as is right now. There are implications to a client of changing the PseudoFS. I have tested moving an export in the PseudoFS with a client mounted. The client will be able to continue accessing the export, though it may see an ESTALE error if it navigates out of the export. The current working directory will go bad and the pwd comment will fail indicating a disconnected mount. I have also seen referencing .. from the root of the export wrapping around back to the root (I believe this is how disconnected mounts are set up). FSAL_PSEUDO lookups and create handles (PUTFH or any use of an NFSv3 handle where the inode isn't cached) which fail during an export update are instead turned into ERR_FSAL_DELAY which turns into NFS4ERR_DELAY or NFS3ERR_JUKEBOX to force the client to retry under the completed update. Change-Id: I507dc17a651936936de82303ff1291677ce136be Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> --- M src/FSAL/FSAL_PSEUDO/handle.c M src/MainNFSD/libganesha_nfsd.ver M src/Protocols/NFS/nfs4_pseudo.c M src/include/export_mgr.h M src/include/nfs_proto_functions.h M src/support/export_mgr.c M src/support/exports.c 7 files changed, 560 insertions(+), 203 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/34/490334/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490334 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I507dc17a651936936de82303ff1291677ce136be Gerrit-Change-Number: 490334 Gerrit-PatchSet: 1 Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com> Gerrit-MessageType: newchange

2 2

Monitoring in Ganesha?
by Bjorn Leffler 22 Nov '21

22 Nov '21

Apart from the counters that you can access through dbus, is there any other monitoring built into Ganesha? I'm thinking of adding it with this higher level plan: - Exporting metrics from Ganesha to Prometheus. - Aggregate data in Prometheus. - Display monitoring consoles and graphs with Grafana. - Package up Prometheus, Grafana and the preconfigured rules/dashboards as a Docker image. - This makes it straightforward to deploy monitoring alongside a Gansha binary. My rough coding plan for the code is to: - Add a USE_MONITORING directive to the CMakeLists.txt files. - Add a build dependency to the Prometheus C client <https://github.com/digitalocean/prometheus-client-c>. - Create a src/monitoring directory for the new source files and templates. - Increment counters and timers throughout the code. - Use histograms to compute latency percentiles, heatmaps, etc. Is this a good idea? Any objections or suggestions? Thanks, Bjorn

3 6

Change in ...nfs-ganesha[next]: ProxyV3: Make the number of sockets configurable.
by Solomon Boulos (GerritHub) 27 Feb '21

27 Feb '21

Solomon Boulos has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/511468 ) Change subject: ProxyV3: Make the number of sockets configurable. ...................................................................... ProxyV3: Make the number of sockets configurable. If you have a lot of latency and enough parallelism, its probably convenient to have lots of backend sockets for the proxy. It was easy enough to move the rpc_init into the config_init for the FSAL. I'm still defaulting it to 32, limiting this to ~1000 (the number of privileged ports is system dependent, but 1000 is a lot). Change-Id: I9f41eb1d5ded5105e7da43cead9085745cc65f0f Signed-off-by: Solomon Boulos <boulos(a)google.com> --- M src/FSAL/FSAL_PROXY_V3/main.c M src/FSAL/FSAL_PROXY_V3/proxyv3_fsal_methods.h M src/FSAL/FSAL_PROXY_V3/rpc.c 3 files changed, 40 insertions(+), 19 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/68/511468/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/511468 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I9f41eb1d5ded5105e7da43cead9085745cc65f0f Gerrit-Change-Number: 511468 Gerrit-PatchSet: 1 Gerrit-Owner: Solomon Boulos <boulos(a)google.com> Gerrit-MessageType: newchange

1 0

Quota sizes
by anirban.chalcogen＠gmail.com 24 Feb '21

24 Feb '21

Hello, So I see a discrepancy in the size of the structures representing quota sizes. Consider struct rquota { int rq_bsize; bool_t rq_active; u_int rq_bhardlimit; u_int rq_bsoftlimit; u_int rq_curblocks; u_int rq_fhardlimit; u_int rq_fsoftlimit; u_int rq_curfiles; u_int rq_btimeleft; u_int rq_ftimeleft; }; They are all 4-byte structures (on most systems). Now, if you look at include/fsal_types.h, they are explicitly defined as 8-byte unsigned. typedef struct fsal_quota__ { uint64_t bhardlimit; uint64_t bsoftlimit; uint64_t curblocks; uint64_t fhardlimit; uint64_t fsoftlimit; uint64_t curfiles; uint64_t btimeleft; uint64_t ftimeleft; uint64_t bsize; } fsal_quota_t; As a result, in rquota_getquota(), when you copy the values from the latter (obtained from underlying fsal) into the former, loss of bits might occur. We are currently seeing this for a customer with a 1024 block size and 4T quota. Do you know if there's a reason (RFC specifications or such) that cause the former to be 32-bytes and if not, would it be OK to increase it? Thanks, Anirban

3 2

Announce Push of V4-dev.52
by Frank Filz 19 Feb '21

19 Feb '21

Branch next Tag:V4-dev.52 Merge Highlights * FSAL_CEPH: override creds when setting sec context on a create * NFS: define a FATTR4_MAX_ATTR_INDEX value Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: 09fafbb Frank S. Filz V4-dev.52 8aec8de Jeff Layton NFS: define a FATTR4_MAX_ATTR_INDEX value 31ff54b Jeff Layton FSAL_CEPH: override creds when setting sec context on a create

1 0

Issue with Ganesha not processing UDP NULL proc requests from AMD
by anirban.chalcogen＠gmail.com 18 Feb '21

18 Feb '21

Hi Daniel or Frank, We spoke about this issue on weekly call just over a week ago. I remember Daniel's advice on that call was to dig into the Ganesha core dump and try to extract the communication channel for each fd from the svc_rqst_epoll_loop() calls and verify that the UDP listen socket for NFS is being epoll'd on. I'm not sure however, the data is present in that stack that would actually allow us to make that call. (gdb) bt full #0 0x00007f3c46616eb3 in epoll_wait () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f3c4876f04c in svc_rqst_epoll_loop (sr_rec=0x4ae8c10) at /<redacted>/libntirpc/src/svc_rqst.c:1013 cc = 0x7f3c46d13eb4 <pthread_cond_timedwait@@GLIBC_2.3.2+516> n = 0x0 ts = {tv_sec = 2750901, tv_nsec = 954488941} timeout_ms = 29000 expire_ms = -1544065341 n_events = 32572 __func__ = "svc_rqst_epoll_loop" #2 0x00007f3c4876f199 in svc_rqst_run_task (wpe=0x4ae8c10) at /<redacted>libntirpc/src/svc_rqst.c:1065 sr_rec = 0x4ae8c10 finished = false __func__ = "svc_rqst_run_task" #3 0x00007f3c48777dca in work_pool_thread (arg=0x7f3c300008c0) at /<redacted>/libntirpc/src/work_pool.c:181 wpt = 0x7f3c300008c0 pool = 0x7f3c48993940 <svc_work_pool> have = 0x0 ts = {tv_sec = 1608803814, tv_nsec = 833467718} rc = 0 spawn = false __func__ = "work_pool_thread" #4 0x00007f3c46d0fea5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f3c466168dd in clone () from /lib64/libc.so.6 No symbol table info available. I am assuming that you would need to chenck the xp_type member in the SVCXPRT structure? However, the members of the latter are lost in the stack since during rearm, we obtain the sr_rec as follows: int svc_rqst_rearm_events(SVCXPRT *xprt) { struct rpc_dplx_rec *rec = REC_XPRT(xprt); struct svc_rqst_rec *sr_rec = (struct svc_rqst_rec *)rec->ev_p; Here, ev_p is a pointer and it seems to me we will not be able to obtain rec from *sr_rec with a container_of() kind macro since ev_p is not being referenced. Do you have another way to ascertain if the FDs polled have the UDP descriptor in them? Thanks, Anirban

2 9

Change in ...nfs-ganesha[next]: NFS: define a FATTR4_MAX_ATTR_INDEX value
by Jeff Layton (GerritHub) 18 Feb '21

18 Feb '21

Jeff Layton has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/511071 ) Change subject: NFS: define a FATTR4_MAX_ATTR_INDEX value ...................................................................... NFS: define a FATTR4_MAX_ATTR_INDEX value Currently a lot of the nfs_proto_tools.c code is set up to use FATTR4_XATTR_SUPPORT as the max defined attribute. That's going to be fragile if we get more new attributes in the future, and it just doesn't read right. Define a new FATTR4_MAX_ATTR_INDEX that is currently set to FATTR4_XATTR_SUPPORT, and use that in the appropriate places in the code. Change-Id: I56cc1a79b72f4409acc90543273323f7013c90ce Signed-off-by: Jeff Layton <jlayton(a)redhat.com> --- M src/Protocols/NFS/nfs_proto_tools.c M src/include/nfsv41.h 2 files changed, 12 insertions(+), 9 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/71/511071/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/511071 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I56cc1a79b72f4409acc90543273323f7013c90ce Gerrit-Change-Number: 511071 Gerrit-PatchSet: 1 Gerrit-Owner: Jeff Layton <jlayton(a)redhat.com> Gerrit-MessageType: newchange

1 0

Conference call may not happen
by Frank Filz 15 Feb '21

15 Feb '21

We just got power back but no internet right now. Hopefully power will hold and Comcast will be up by the morning. If not I may try and dial into the call. Sent from my iPhone

1 0

Announce Push of V4-dev.51
by Frank Filz 12 Feb '21

12 Feb '21

Branch next Tag:V4-dev.51 Merge Highlights * FSAL_CEPH: use ceph_ll_lookup_vino if available * some valid attr cleanup, especially for SEC_LABEL * NLM: Don't allow locks on non-REGULAR_FILE Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: 4df403a Frank S. Filz V4-dev.51 2bae2fd Frank S. Filz NLM: Don't allow locks on non-REGULAR_FILE c22da93 Jeff Layton nfs4: fix bad attr comparison in xdr_fattr4_encode 6849ce7 Jeff Layton nfs4: encode only valid attrs 6abb9f1 Jeff Layton mdcache: only query for seclabel if it was requested 2f388cd Jeff Layton FSAL_CEPH: use ceph_ll_lookup_vino if available

1 0

Change in ...nfs-ganesha[next]: FSAL_CEPH: override creds when setting sec context on a create
by Jeff Layton (GerritHub) 12 Feb '21

12 Feb '21

Jeff Layton has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/510758 ) Change subject: FSAL_CEPH: override creds when setting sec context on a create ...................................................................... FSAL_CEPH: override creds when setting sec context on a create I was seeing problems creating files with the mode 0444 with security labels enabled. The create would work, but the subsequent setxattr to set the security label would fail, and that would fail the operation altogether. Use the fsal_private field in the op_ctx to pass a flag to setattr2 that tells it to use root creds for the setxattr. Only set that flag after a create. Change-Id: I3e9089eb18188f4d36a5f9c5fba48755b9d086cc Signed-off-by: Jeff Layton <jlayton(a)redhat.com> --- M src/FSAL/FSAL_CEPH/handle.c 1 file changed, 27 insertions(+), 2 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/58/510758/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/510758 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I3e9089eb18188f4d36a5f9c5fba48755b9d086cc Gerrit-Change-Number: 510758 Gerrit-PatchSet: 1 Gerrit-Owner: Jeff Layton <jlayton(a)redhat.com> Gerrit-MessageType: newchange

1 0