February 2023 - Devel - NFS Ganesha List Archives

Change in ...nfs-ganesha[next]: Allow EXPORT pseudo path to be changed during export update
by Frank Filz (GerritHub) 19 Feb '24

19 Feb '24

Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490334 ) Change subject: Allow EXPORT pseudo path to be changed during export update ...................................................................... Allow EXPORT pseudo path to be changed during export update This also fully allows adding or removing NFSv4 support from an export since we can now handle the PseudoFS swizzing that occurs. Note that an explicit PseudoFS export may be removed or added, though you can not change it from export_id 0 because we currently don't allow changing the export_id. Note that this patch doesn't handle DBUS add or remove export though that is an option to improve. I may add them to this patch (it wouldn't be that hard) but I want to get this reviewed as is right now. There are implications to a client of changing the PseudoFS. I have tested moving an export in the PseudoFS with a client mounted. The client will be able to continue accessing the export, though it may see an ESTALE error if it navigates out of the export. The current working directory will go bad and the pwd comment will fail indicating a disconnected mount. I have also seen referencing .. from the root of the export wrapping around back to the root (I believe this is how disconnected mounts are set up). FSAL_PSEUDO lookups and create handles (PUTFH or any use of an NFSv3 handle where the inode isn't cached) which fail during an export update are instead turned into ERR_FSAL_DELAY which turns into NFS4ERR_DELAY or NFS3ERR_JUKEBOX to force the client to retry under the completed update. Change-Id: I507dc17a651936936de82303ff1291677ce136be Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> --- M src/FSAL/FSAL_PSEUDO/handle.c M src/MainNFSD/libganesha_nfsd.ver M src/Protocols/NFS/nfs4_pseudo.c M src/include/export_mgr.h M src/include/nfs_proto_functions.h M src/support/export_mgr.c M src/support/exports.c 7 files changed, 560 insertions(+), 203 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/34/490334/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490334 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I507dc17a651936936de82303ff1291677ce136be Gerrit-Change-Number: 490334 Gerrit-PatchSet: 1 Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com> Gerrit-MessageType: newchange

2 2

Help needed on FD
by Alok Sinha 31 Jan '24

31 Jan '24

FD created at below path does not have correct link in /proc/self/fd. With vfs_readlink_by_handle , I can get a relative path but cannot get the absolute path of the FD. I need a suggestion for one of the below to get around a production bug: - How to get the full path of the FD? - How to get the parent vfs_fsal_obj_handle? - Anyway to bypass this flow by changing any config file? #0 vfs_create_handle (exp_hdl=0x55801aca22c0, hdl_desc=0x7f93889b1600, handle=0x7f93889b13f8, attrs_out=0x7f93889b1430) at /home/alok/pub/splfs-cache-2.8.3/src/FSAL/FSAL_VFS/handle.c:2020 #1 0x00007f94a1f0e13b in mdcache_locate_host (fh_desc=0x7f93889b1600, export=0x55801aca1f20, entry=0x7f93889b1578, attrs_out=0x0) at /home/alok/pub/splfs-cache-2.8.3/src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:1109 #2 0x00007f94a1f02455 in mdcache_create_handle (exp_hdl=0x55801aca1f20, fh_desc=0x7f93889b1600, handle=0x7f93889b15d8, attrs_out=0x0) at /home/alok/pub/splfs-cache-2.8.3/src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c:1613 #3 0x00007f94a1ec5ead in nfs4_mds_putfh (data=0x7f93201119e0) at /home/alok/pub/splfs-cache-2.8.3/src/Protocols/NFS/nfs4_op_putfh.c:211 #4 0x00007f94a1ec60d3 in nfs4_op_putfh (op=0x7f93201186d0, data=0x7f93201119e0, resp=0x7f932010ab20) at /home/alok/pub/splfs-cache-2.8.3/src/Protocols/NFS/nfs4_op_putfh.c:281 #5 0x00007f94a1ea96ae in process_one_op (data=0x7f93201119e0, status=0x7f93889b1b48) at /home/alok/pub/splfs-cache-2.8.3/src/Protocols/NFS/nfs4_Compound.c:920 #6 0x00007f94a1eaa8c5 in nfs4_Compound (arg=0x7f9320138d78, req=0x7f93201385a0, res=0x7f9320106840) at /home/alok/pub/splfs-cache-2.8.3/src/Protocols/NFS/nfs4_Compound.c:1329 #7 0x00007f94a1de8567 in nfs_rpc_process_request (reqdata=0x7f93201385a0) ---Type <return> to continue, or q <return> to quit--- at /home/alok/pub/splfs-cache-2.8.3/src/MainNFSD/nfs_worker_thread.c:1484 #8 0x00007f94a1de8854 in nfs_rpc_valid_NFS (req=0x7f93201385a0) at /home/alok/pub/splfs-cache-2.8.3/src/MainNFSD/nfs_worker_thread.c:1591 #9 0x00007f94a13b6faf in svc_vc_decode (req=0x7f93201385a0) at /home/alok/pub/splfs-cache-2.8.3/src/libntirpc/src/svc_vc.c:829 #10 0x00007f94a13b3225 in svc_request (xprt=0x7f9320000ce0, xdrs=0x7f9320066260) at /home/alok/pub/splfs-cache-2.8.3/src/libntirpc/src/svc_rqst.c:793 #11 0x00007f94a13b6ec0 in svc_vc_recv (xprt=0x7f9320000ce0) at /home/alok/pub/splfs-cache-2.8.3/src/libntirpc/src/svc_vc.c:802 #12 0x00007f94a13b31a5 in svc_rqst_xprt_task (wpe=0x7f9320000f00) at /home/alok/pub/splfs-cache-2.8.3/src/libntirpc/src/svc_rqst.c:774 #13 0x00007f94a13bcab0 in work_pool_thread (arg=0x7f9218003340) at /home/alok/pub/splfs-cache-2.8.3/src/libntirpc/src/work_pool.c:184 #14 0x00007f94a1badea5 in start_thread () from /lib64/libpthread.so.0 #15 0x00007f94a16ce9fd in clone () from /lib64/libc.so.6 -- Alok Sinha www.spillbox.io https://youtu.be/U-YupjLQ9bU

2 2

[S] Change in ...nfs-ganesha[next]: config: Fix copy/paste error in %include handling
by Martin Schwenke (GerritHub) 28 Feb '23

28 Feb '23

Martin Schwenke has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/550319 ) Change subject: config: Fix copy/paste error in %include handling ...................................................................... config: Fix copy/paste error in %include handling Signed-off-by: Martin Schwenke <mschwenke(a)ddn.com> Change-Id: I5237f8f650da2989ef46dfed8623587bf523d9c5 --- M src/config_parsing/conf_lex.l 1 file changed, 14 insertions(+), 4 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/19/550319/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/550319 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I5237f8f650da2989ef46dfed8623587bf523d9c5 Gerrit-Change-Number: 550319 Gerrit-PatchSet: 1 Gerrit-Owner: Martin Schwenke <martin(a)meltin.net> Gerrit-MessageType: newchange

1 0

[S] Change in ...nfs-ganesha[next]: recovery: Fix typo in log messages
by Martin Schwenke (GerritHub) 28 Feb '23

28 Feb '23

Martin Schwenke has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/550299 ) Change subject: recovery: Fix typo in log messages ...................................................................... recovery: Fix typo in log messages Signed-off-by: Martin Schwenke <mschwenke(a)ddn.com> Change-Id: I5e77b70c5ebb5bd55c9391755ab392de85bef9cb --- M src/SAL/recovery/recovery_fs.c 1 file changed, 12 insertions(+), 2 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/99/550299/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/550299 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I5e77b70c5ebb5bd55c9391755ab392de85bef9cb Gerrit-Change-Number: 550299 Gerrit-PatchSet: 1 Gerrit-Owner: Martin Schwenke <martin(a)meltin.net> Gerrit-MessageType: newchange

1 0

[S] Change in ...nfs-ganesha[next]: docs: Switch GitHub git URLs to https
by Martin Schwenke (GerritHub) 28 Feb '23

28 Feb '23

Martin Schwenke has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/550318 ) Change subject: docs: Switch GitHub git URLs to https ...................................................................... docs: Switch GitHub git URLs to https GitHub no longer appears to support git: URLs for git access. Signed-off-by: Martin Schwenke <mschwenke(a)ddn.com> Change-Id: I01212e2177fe3e2fc51e76940291290e4ed41b0b --- M src/COMPILING_HOWTO.txt M src/CONTRIBUTING_HOWTO.txt 2 files changed, 15 insertions(+), 3 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/18/550318/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/550318 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I01212e2177fe3e2fc51e76940291290e4ed41b0b Gerrit-Change-Number: 550318 Gerrit-PatchSet: 1 Gerrit-Owner: Martin Schwenke <martin(a)meltin.net> Gerrit-MessageType: newchange

1 0

Announce Push of V5-dev.2
by Frank Filz 23 Feb '23

23 Feb '23

Branch next Tag:V5-dev.2 Merge Highlights * Add Prometheus and Grafana monitoring stack. Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: 7445b0ef6 Frank S. Filz V5-dev.2 b6c59df5a Bjorn Leffler Add Prometheus and Grafana monitoring stack.

1 0

lock failure
by Alok Sinha 23 Feb '23

23 Feb '23

What could be the problem? This is with version 2.8.3. spillbox-nfs-server-01 : ganesha.nfsd.regress-vcsregrfeb22-nfs-server-19310[svc_640] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed to create new lock owner Lock: obj=0x2b8a9406f3b8, fileid=83243891, type=READ , start=0x0, end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4 0x2b89b0387530: clientid={0x2b893c30cf90 ClientID={Epoch=0x63f5d208 Counter=0x000000b8} CONFIRMED Client={0x2b893c067760 name=(39:Linux NFSv4.0 spillbox-33/ 10.178.28.118) refcount=1} t_delta=0 reservations=1 refcount=10 cb_prog=1073741824 r_addr=10.178.28.54.177.211 r_netid=tcp} owner=(24:0x6f70656e2069643a00000042000000000002b0d23b581e0d) confirmed=1 seqid=125711 refcount=130} 23/02/2023 18:08:28 : epoch 63f5d208 : spillbox-nfs-server-01 : ganesha.nfsd.regress-vcsregrfeb22-nfs-server-19310[svc_628] create_nfs4_owner :NFS4 LOCK :CRIT :Related {STATE_OPEN_OWNER_NFSV4 0x2b89f02a7f80: clientid={0x2b8930343fb0 ClientID={Epoch=0x63f5d208 Counter=0x000000c9} CONFIRMED Client={0x2b89300bd880 name=(39:Linux NFSv4.0 spillbox-80/10.178.28.118) refcount=1} t_delta=0 reservations=1 refcount=19 cb_prog=1073741824 r_addr=10.178.28.23.155.163 r_netid=tcp} owner=(24:0x6f70656e2069643a00000042000000000002b1a675f4638c) confirmed=1 seqid=127246 refcount=121} doesn't match for {STATE_LOCK_OWNER_NFSV4 0x2b89783ea1c0: clientid={0x2b8930343fb0 ClientID={Epoch=0x63f5d208 Counter=0x000000c9} CONFIRMED Client={0x2b89300bd880 name=(39:Linux NFSv4.0 spillbox-80/10.178.28.118) refcount=1} t_delta=0 reservations=1 refcount=19 cb_prog=1073741824 r_addr=10.178.28.23.155.163 r_netid=tcp} owner=(20:0x6c6f636b2069643a000000420000000000000000) confirmed=1 seqid=0 related_owner={STATE_OPEN_OWNER_NFSV4 0x2b88c81bada0: clientid={0x2b8930343fb0 ClientID={Epoch=0x63f5d208 Counter=0x000000c9} CONFIRMED Client={0x2b89300bd880 name=(39:Linux NFSv4.0 spillbox-80/ 10.178.28.118) refcount=1} t_delta=0 reservations=1 refcount=19 cb_prog=1073741824 r_addr=10.178.28.23.155.163 r_netid=tcp} owner=(24:0x6f70656e2069643a00000042000000000002af455610f805) confirmed=1 seqid=109320 refcount=187} refcount=3} -- Alok Sinha www.spillbox.io https://www.youtube.com/watch?v=g3RujrjlZIY

1 0

NFSv3 CREATE op is breaking RFC1813
by Sagar Singh 23 Feb '23

23 Feb '23

NFSv3 CREATE is returning NFS3ERR_PERM on permission error, which seems to breaking the RFC as all the return errors for CREATE: ERRORS NFS3ERR_IO NFS3ERR_ACCES NFS3ERR_EXIST NFS3ERR_NOTDIR NFS3ERR_NOSPC NFS3ERR_ROFS NFS3ERR_NAMETOOLONG NFS3ERR_DQUOT NFS3ERR_STALE NFS3ERR_BADHANDLE NFS3ERR_NOTSUPP NFS3ERR_SERVERFAULT We should have returned NFS3ERR_ACCES, instead of NFS3ERR_PERM. Although in NFSv4 CREATE, NFS3ERR_PERM is valid. What are your thoughts? Patch introducing this breakage: https://github.com/nfs-ganesha/nfs-ganesha/commit/edba4a5c1be0178c0c32e9c87… Thanks, Sagar Singh

1 0

Open FD Count does not decrease & new file creation fails
by Ankith Acharya 22 Feb '23

22 Feb '23

Hello I have long running test which keeps failinging after 2-3 days. There is workload which runs fine for 2-3 days and then Ganesha process completely hangs up for file creation. Mounting, unmounting, directory listing, file read - work. However, new file creation fails completely. The process does not allow file creation until it restarts. I have waited for 2-3 days to check if process recovers but it does not. Ganesha version : 3.5 Client : Windows 2016 NFS version : 3 VFS : custom Workload Concurrent access from multiple threads in single client. 1 thread continuously (in a loop) running python os.walk (i.e., readdir) of the entire filesystem, roughly ~5M files total. Client runs robocopy command copying folder over to NFS share with 10 threads. The folder contains randomly between 1-1000 files of random size between 1Kb-50MB. When the writes complete, a single thread verifies written content, then deletes it. The robocopy continues again. Debugging - I looked at the logs and debugger and file creation fails as `mdcache_lru_fds_available` evaluates to false everytime. (https://github.com/nfs-ganesha/nfs-ganesha/blob/V3.5/src/FSAL/Stackable_FSA…) - I checked the "lru_state" variable and "open_fd_count" has hit the limit. So, ganesha sends `ERR_FSAL_DELAY` response back each time. (gdb) p lru_state $1 = {entries_hiwat = 500000, entries_used = 500000, entries_release_size = 100, chunks_hiwat = 25000, chunks_used = 610, fds_system_imposed = 400000, fds_hard_limit = 396000, fds_hiwat = 360000, fds_lowat = 200000, futility = 1, per_lane_work = 50, biggest_window = 160000, prev_fd_count = 396000, prev_time = 1676657325, fd_state = 3} - However, this situation keeps continues indefinitely. The "lru_thread" is actively trying to reap but it is not making any progress. Debug logs prove that. I can probably increase the limits for open fd but it just feels it will hit this limit in a while. lru_run :INODE LRU :F_DBG :formeropen=396000 totalwork=0 workpass=37 totalclosed:0 Actually processed 6 entries on lane 8 closing 0 descriptors - Even unmounting the client does not help. I have ready server in this state. I can run any debug commands and provide logs. Thanks in advance for the help.

1 0

Is there a plan to support RDMA?
by liuchong.8＠163.com 21 Feb '23

21 Feb '23

Hi everybody, If we would plan to support RDMA in latest developing version?

2 1