March 2020 - Devel - NFS Ganesha List Archives

Change in ...nfs-ganesha[next]: MDCACHE - Add MDCACHE {} config block
by Daniel Gryniewicz (GerritHub) 05 Nov '20

05 Nov '20

Daniel Gryniewicz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/454929 Change subject: MDCACHE - Add MDCACHE {} config block ...................................................................... MDCACHE - Add MDCACHE {} config block Add a config block name MDCACHE that is a copy of CACHEINODE. Both can be configured, but MDCACHE will override CACHEINODE. This allows us to deprecate CACHEINODE. Change-Id: I49012723132ae6105b904a60d1a96bb2bf78d51b Signed-off-by: Daniel Gryniewicz <dang(a)fprintf.net> --- M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_read_conf.c M src/config_samples/ceph.conf M src/config_samples/config.txt M src/config_samples/ganesha.conf.example M src/doc/man/ganesha-cache-config.rst M src/doc/man/ganesha-config.rst 6 files changed, 31 insertions(+), 7 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/29/454929/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/454929 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I49012723132ae6105b904a60d1a96bb2bf78d51b Gerrit-Change-Number: 454929 Gerrit-PatchSet: 1 Gerrit-Owner: Daniel Gryniewicz <dang(a)redhat.com> Gerrit-MessageType: newchange

3 2

lseek gets bad offset from nfs client with ganesha/gluster which supports SEEK
by Kinglong Mee 14 Sep '20

14 Sep '20

The latest ganesha/gluster supports seek according to, https://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-41#section-15.11 From the given sa_offset, find the next data_content4 of type sa_what in the file. If the server can not find a corresponding sa_what, then the status will still be NFS4_OK, but sr_eof would be TRUE. If the server can find the sa_what, then the sr_offset is the start of that content. If the sa_offset is beyond the end of the file, then SEEK MUST return NFS4ERR_NXIO. For a file's filemap as, Part 1: HOLE 0x0000000000000000 ---> 0x0000000000600000 Part 2: DATA 0x0000000000600000 ---> 0x0000000000700000 Part 3: HOLE 0x0000000000700000 ---> 0x0000000001000000 SEEK(0x700000, SEEK_DATA) gets result (sr_eof:1, sr_offset:0x70000) from ganesha/gluster; SEEK(0x700000, SEEK_HOLE) gets result (sr_eof:0, sr_offset:0x70000) from ganesha/gluster. If an application depends the lseek result for data searching, it may enter infinite loop. while (1) { next_pos = lseek(fd, cur_pos, seek_type); if (seek_type == SEEK_DATA) { seek_type = SEEK_HOLE; } else { seek_type = SEEK_DATA; } if (next_pos == -1) { return ; cur_pos = next_pos; } The lseek syscall always gets 0x70000 from nfs client for those two cases, but, if underlying filesystem is ext4/f2fs, or the nfs server is knfsd, the lseek(0x700000, SEEK_DATA) gets ENXIO. I wanna to know, should I fix the ganesha/gluster as knfsd return ENXIO for the first case? or should I fix the nfs client to return ENXIO for the first case? thanks, Kinglong Mee

4 8

Crash in dupreq
by katcherw＠gmail.com 03 Apr '20

03 Apr '20

We saw a crash in version 2.8.2 running an iozone crash: Program terminated with signal 11, Segmentation fault. #0 0x00007ff028dce240 in atomic_sub_uint32_t (var=0x104, sub=1) at /usr/src/debug/nfs-ganesha-2.8.2/include/abstract_atomic.h:384 384 return __atomic_sub_fetch(var, sub, __ATOMIC_SEQ_CST); Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-12.el7.x86_64 dbus-libs-1.11.16-1.fc26.x86_64 elfutils-libelf-0.158-3.el7.x86_64 elfutils-libs-0.158-3.el7.x86_64 glibc-2.17-55.el7.x86_64 gssproxy-0.3.0-9.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-8.el7.x86_64 libacl-2.2.51-12.el7.x86_64 libattr-2.4.46-12.el7.x86_64 libblkid-2.23.2-16.el7.x86_64 libcap-2.22-8.el7.x86_64 libcom_err-1.42.9-10.el7.x86_64 libgcrypt-1.5.3-4.el7.x86_64 libgpg-error-1.12-3.el7.x86_64 libnfsidmap-0.25-9.el7.x86_64 libselinux-2.5-11.el7.x86_64 libuuid-2.23.2-16.el7.x86_64 libwbclient-4.8.3-4.el7.x86_64 pcre-8.32-12.el7.x86_64 samba-client-libs-4.8.3-4.el7.x86_64 samba-winbind-modules-4.8.3-4.el7.x86_64 sssd-client-1.11.2-65.el7.x86_64 systemd-libs-219-42.el7_4.1.x86_64 userspace-rcu-0.7.16-1.el7.x86_64 xz-libs-5.1.2-8alpha.el7.x86_64 zlib-1.2.7-13.el7.x86_64 (gdb) bt #0 0x00007ff028dce240 in atomic_sub_uint32_t (var=0x104, sub=1) at /usr/src/debug/nfs-ganesha-2.8.2/include/abstract_atomic.h:384 #1 0x00007ff028dce265 in atomic_dec_uint32_t (var=0x104) at /usr/src/debug/nfs-ganesha-2.8.2/include/abstract_atomic.h:405 #2 0x00007ff028dd1b65 in dupreq_entry_put (dv=0x0) at /usr/src/debug/nfs-ganesha-2.8.2/RPCAL/nfs_dupreq.c:912 #3 0x00007ff028dd4630 in nfs_dupreq_rele (req=0x7fec44a269f0, func=0x7ff029159d60 <nfs3_func_desc+672>) at /usr/src/debug/nfs-ganesha-2.8.2/RPCAL/nfs_dupreq.c:1382 #4 0x00007ff028d74bc5 in free_args (reqdata=0x7fec44a269f0) at /usr/src/debug/nfs-ganesha-2.8.2/MainNFSD/nfs_worker_thread.c:693 #5 0x00007ff028d77783 in nfs_rpc_process_request (reqdata=0x7fec44a269f0) at /usr/src/debug/nfs-ganesha-2.8.2/MainNFSD/nfs_worker_thread.c:1509 #6 0x00007ff028d77a74 in nfs_rpc_valid_NFS (req=0x7fec44a269f0) at /usr/src/debug/nfs-ganesha-2.8.2/MainNFSD/nfs_worker_thread.c:1601 #7 0x00007ff028b2632d in svc_vc_decode (req=0x7fec44a269f0) at /usr/src/debug/nfs-ganesha-2.8.2/libntirpc/src/svc_vc.c:829 #8 0x00007ff028b227bf in svc_request (xprt=0x7fee0c202ad0, xdrs=0x7fec3c987a70) at /usr/src/debug/nfs-ganesha-2.8.2/libntirpc/src/svc_rqst.c:793 #9 0x00007ff028b2623e in svc_vc_recv (xprt=0x7fee0c202ad0) at /usr/src/debug/nfs-ganesha-2.8.2/libntirpc/src/svc_vc.c:802 #10 0x00007ff028b22740 in svc_rqst_xprt_task (wpe=0x7fee0c202cf0) at /usr/src/debug/nfs-ganesha-2.8.2/libntirpc/src/svc_rqst.c:774 #11 0x00007ff028b23048 in svc_rqst_epoll_loop (wpe=0x108d570) at /usr/src/debug/nfs-ganesha-2.8.2/libntirpc/src/svc_rqst.c:1089 #12 0x00007ff028b2baff in work_pool_thread (arg=0x7fed44052070) at /usr/src/debug/nfs-ganesha-2.8.2/libntirpc/src/work_pool.c:184 #13 0x00007ff0270c8df3 in start_thread () from /lib64/libpthread.so.0 #14 0x00007ff0267cd3dd in clone () from /lib64/libc.so.6 (gdb) frame 5 #5 0x00007ff028d77783 in nfs_rpc_process_request (reqdata=0x7fec44a269f0) at /usr/src/debug/nfs-ganesha-2.8.2/MainNFSD/nfs_worker_thread.c:1509 1509 free_args(reqdata); (gdb) p *reqdata $2 = { svc = { rq_xprt = 0x7fee0c202ad0, rq_clntname = 0x0, rq_svcname = 0x0, rq_xdrs = 0x7fec3c987a70, rq_u1 = 0x0, rq_u2 = 0x0, rq_cksum = 14781944753697519387, rq_auth = 0x7ff028d45cd0 <svc_auth_none>, rq_ap1 = 0x0, rq_ap2 = 0x0, ... Notice rq_u1 == 0. nfs_rpc_process_request is calling free_args which is calling nfs_dupreq_rele. There is a comment at the beginning of nfs_dupreq_rele: * We assert req->rq_u1 now points to the corresponding duplicate request * cache entry (dv). But this assumption doesn't seem to be true, and eventually this will be dereferenced leading to a crash. We're trying to reproduce the problem with NIV_FULL_DEBUG. Has anyone seen this problem before, or know what would cause rq_u1 to be 0? Should there be a check for NULL in nfs_dupreq_rele in addition to DUPREQ_NOCACHE? Bill

6 7

method to use client_mgr.c and server_stats.c in NFS Ganesha
by Suresh Kosuru 03 Apr '20

03 Apr '20

Hi, Can you point me help? how can I use client_mgr.c and server_stats.c in nfs ganesha and where will this info logging? Thanks Suresh

3 2

Announce Push of V4-dev.10
by Frank Filz 27 Mar '20

27 Mar '20

Branch next Tag:V4-dev.10 Merge Highlights * selinux: new policy to allow gluster (glusterd) to start ganesha * packaging: typos in nfs-ganesha.spec.in.cmake * In mdc_open2_by_name() after lookup check if obj_handle.type==REGULAR_FILE * MDCACHE - Close unexport race for used entries Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: 091816d Frank S. Filz V4-dev.10 e066494 Daniel Gryniewicz MDCACHE - Close unexport race for used entries 7dd4538 Madhu Thorat In mdc_open2_by_name() after lookup check if obj_handle.type==REGULAR_FILE 6a1e149 Kaleb S. KEITHLEY packaging: typos in nfs-ganesha.spec.in.cmake 8c31aa2 Kaleb S. KEITHLEY selinux: new policy to allow gluster (glusterd) to start ganesha

1 0

Change in ...nfs-ganesha[next]: packaging: typos in nfs-ganesha.spec.in.cmake
by Kaleb KEITHLEY (GerritHub) 27 Mar '20

27 Mar '20

Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/488232 ) Change subject: packaging: typos in nfs-ganesha.spec.in.cmake ...................................................................... packaging: typos in nfs-ganesha.spec.in.cmake $subject Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> Change-Id: Ifc3b3a34f45e1a440486cd5325bd3eebe4dcfe65 --- M src/nfs-ganesha.spec-in.cmake 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/32/488232/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/488232 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Ifc3b3a34f45e1a440486cd5325bd3eebe4dcfe65 Gerrit-Change-Number: 488232 Gerrit-PatchSet: 1 Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: Update dupreq after svc_sendreply failure
by Malahal (GerritHub) 27 Mar '20

27 Mar '20

Malahal has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/488188 ) Change subject: Update dupreq after svc_sendreply failure ...................................................................... Update dupreq after svc_sendreply failure If svc_sendreply() fails, we don't update the dupreq. The dupreq state will be forever in DUPREQ_START! Here after, all retries will be dropped as the dupreq is found in a processing state leading to endless retries. svc_sendreply() can fail in UDP transports. TCP transports don't fail unless there is some kind of RPC encoding failure. Change-Id: Ib2b0bdd689d57c141448ccfb7c0789f3dfd6fa9c Signed-off-by: Malahal Naineni <malahal(a)us.ibm.com> --- M src/MainNFSD/nfs_worker_thread.c 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/88/488188/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/488188 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Ib2b0bdd689d57c141448ccfb7c0789f3dfd6fa9c Gerrit-Change-Number: 488188 Gerrit-PatchSet: 1 Gerrit-Owner: Malahal <malahal(a)gmail.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: In mdc_open2_by_name() after lookup check if obj_handle.type==REGULAR...
by Madhu Thorat (GerritHub) 26 Mar '20

26 Mar '20

Madhu Thorat has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/488074 ) Change subject: In mdc_open2_by_name() after lookup check if obj_handle.type==REGULAR_FILE ...................................................................... In mdc_open2_by_name() after lookup check if obj_handle.type==REGULAR_FILE In mdc_open2_by_name() after mdc_lookup() completes successfully and prior to calling FSALs open2(..) we don't check the entry->obj_handle.type to verify if the object is for a regular file. It may be possible that prior to calling mdc_lookup() in mdc_open2_by_name(), another thread working on a different client request may have created a symbolic link with the same name. Using an object created for a symbolic link in nfs4_op_open() code path lead to a crash. Fixed this by adding a check in mdc_open2_by_name() to verify entry->obj_handle.type is REGULAR_FILE and then proceed to call a FSALs open2(..). Change-Id: Ib86b53f1c7bb6652509ef98575f6b1e8ff2fde82 Signed-off-by: Madhu Thorat <madhu.punjabi(a)in.ibm.com> --- M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_file.c 1 file changed, 13 insertions(+), 0 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/74/488074/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/488074 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Ib86b53f1c7bb6652509ef98575f6b1e8ff2fde82 Gerrit-Change-Number: 488074 Gerrit-PatchSet: 1 Gerrit-Owner: Madhu Thorat <madhu.punjabi(a)in.ibm.com> Gerrit-MessageType: newchange

1 0

Issues with Changing pseudo path of an export
by Frank Filz 24 Mar '20

24 Mar '20

So some interesting issues arise if we change the pseudo path of an export or change the NFS versions supported. What happens to state attached to files where the client will now get a stale file handle? This may not be an issue if the Pseudo Path to an export changes, but certainly would be a problem if v3 or v4 were dropped from the export. If it really is only a problem for dropping v3 or v4 we could do the version specific state cleanup for the export. Frank

4 4

Improving mdcache_lru_cleanup_try_push
by Ashish Sangwan 24 Mar '20

24 Mar '20

Hi Daniel, It looks like mdcache_lru_cleanup_try_push() can be improved for the cases when there are ops executing parallely with an unexport request. In this case mdcache_lru_cleanup_try_push() is not able to drop the sentinel ref and mark the entry for cleanup. This is because one extra ref is taken by the op which is currently executing. The total number of refs are > 2. Once the current op execution finishes (after the execution of mdcache_unexport()) the sentinel ref remains and the entry is also not marked for cleanup even though no export is referring to it (entry->first_export_id == -1). Is it possible to modify mdcache_lru_cleanup_try_push() such that even if the refcnt is > 2 but if the entry->first_export_id == -1 (the current export which is going away was the only one referring to this entry), we should drop the sentinel ref and mark the entry for cleanup? This will ensure we cleanup the entries when the current op does put_ref and these are reused faster than other mdcache entries which are actually pointing to valid exports. Thanks, Ashish

2 6