April 2020 - Devel - NFS Ganesha List Archives

Announce Push of V4-dev.13
by Frank Filz 17 Apr '20

17 Apr '20

Branch next Tag:V4-dev.13 Merge Highlights * Do lock_list empty check under the state lock. * selinux: move external types to a require{} block * Avoid acquiring the state_lock recursively in some code paths Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: 5731127 Frank S. Filz V4-dev.13 8c9c742 Madhu Thorat Avoid acquiring the state_lock recursively in some code paths f7ecb79 Kaleb S. KEITHLEY selinux: move external types to a require{} block 59142bb Malahal Naineni Do lock_list empty check under the state lock.

1 0

Issues seen with Blocked Locks
by Trishali Nayar 14 Apr '20

14 Apr '20

Hi all, We have seen multiple crashes (with and without ASAN) Posting the stack traces for reference... Crash 1: (gdb) bt #0 0x00007f6d41f3523b in raise () from /lib64/libpthread.so.0 #1 0x000000000047a9f0 in crash_handler (signo=6, info=0x7f6d1db0a030, ctx=0x7f6d1db09f00) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/MainNFSD/nfs_init.c:244 #2 <signal handler called> #3 0x00007f6d417791d7 in raise () from /lib64/libc.so.6 #4 0x00007f6d4177a8c8 in abort () from /lib64/libc.so.6 #5 0x00007f6d43cb8e6e in __sanitizer::Abort() () from /lib64/libasan.so.5 #6 0x00007f6d43cc1558 in __sanitizer::Die() () from /lib64/libasan.so.5 #7 0x00007f6d43ca4b15 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) () from /lib64/libasan.so.5 #8 0x00007f6d43ca5567 in __asan_report_load8 () from /lib64/libasan.so.5 #9 0x00000000005c63ca in process_blocked_lock_upcall (block_data=0x60c0007feac0) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/SAL/state_lock.c:1809 #10 0x00000000005ba962 in state_blocked_lock_caller (ctx=0x616000a6fe80) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/SAL/state_async.c:81 #11 0x0000000000688479 in fridgethr_start_routine (arg=0x616000a6fe80) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/support/fridgethr.c:554 #12 0x00007f6d41f2ddc5 in start_thread () from /lib64/libpthread.so.0 #13 0x00007f6d4183b73d in clone () from /lib64/libc.so.6 (gdb) f 9 #9 0x00000000005c63ca in process_blocked_lock_upcall (block_data=0x60c0007feac0) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/SAL/state_lock.c:1809 1809 PTHREAD_RWLOCK_unlock(&lock_entry->sle_obj->state_hdl-> state_lock); (gdb) print *block_data $1 = {sbd_list = {next = 0x20000011, prev = 0x0}, sbd_grant_type = STATE_GRANT_POLL, sbd_block_type = STATE_BLOCK_POLL, sbd_granted_callback = 0x57afd2 <nlm_granted_callback>, sbd_blocked_cookie = 0x0, sbd_lock_entry = 0x6110007a1240, sbd_prot = {sbd_nlm = { sbd_nlm_fh = {n_len = 56, n_bytes = 0x60c0007feb00 "C"}, sbd_nlm_fh_buf = "C\000\000\004\060\060\000\n\000\002\000(\000\300 \250z\254^%j\365\002\000\000\002\000\000\000\000\251\344\000\000\000\000 \000\000\000\000\000\000=k@y\273\344", '\000' <repeats 16 times>}, sbd_v4 = 0x38}} (gdb) print *(state_lock_entry_t *)0x6110007a1240 $2 = {sle_list = {next = 0x7400001a, prev = 0x0}, sle_owner_locks = {next = 0x0, prev = 0x0}, sle_client_locks = {next = 0x0, prev = 0x0}, sle_state_locks = {next = 0x0, prev = 0x0}, sle_export_locks = {next = 0x0, prev = 0x0}, sle_export = 0x615000000348, sle_obj = 0x6190013ce5b8, sle_block_data = 0x60c0007feac0, sle_owner = 0x0, sle_state = 0x61200159cc40, sle_blocked = STATE_CANCELED, sle_ref_count = 0, sle_lock = { <<< cancelled lock_sle_type = FSAL_POSIX_LOCK, lock_type = FSAL_LOCK_W, lock_start = 527338800, lock_length = 517175, lock_reclaim = false}, sle_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = -1, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\377\377\377\377", '\000' <repeats 19 times>, __align = 0}} (gdb) The problem is seen when doing unlock using lock_entry and ASAN reports heap-use-after-free. try_to_grant_lock() finds cancelled block entry and invokes remove_from_locklist() which calls lock_entry_dec_ref() causing free. ************************************************************************************************************************************************************************************ Crash 2: (gdb) bt #0 0x00007f212862923b in raise () from /lib64/libpthread.so.0 #1 0x000000000047a9f0 in crash_handler (signo=6, info=0x7f210bf6d070, ctx=0x7f210bf6cf40) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/MainNFSD/nfs_init.c:244 #2 <signal handler called> #3 0x00007f2127e6d1d7 in raise () from /lib64/libc.so.6 #4 0x00007f2127e6e8c8 in abort () from /lib64/libc.so.6 #5 0x00007f212a3ace6e in __sanitizer::Abort() () from /lib64/libasan.so.5 #6 0x00007f212a3b5558 in __sanitizer::Die() () from /lib64/libasan.so.5 #7 0x00007f212a398b15 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) () from /lib64/libasan.so.5 #8 0x00007f212a399567 in __asan_report_load8 () from /lib64/libasan.so.5 #9 0x00000000005ba881 in state_blocked_lock_caller (ctx=0x616000530480) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/SAL/state_async.c:72 #10 0x0000000000688479 in fridgethr_start_routine (arg=0x616000530480) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/support/fridgethr.c:554 #11 0x00007f2128621dc5 in start_thread () from /lib64/libpthread.so.0 #12 0x00007f2127f2f73d in clone () from /lib64/libc.so.6 (gdb) (gdb) f 9 #9 0x00000000005ba881 in state_blocked_lock_caller (ctx=0x616000530480) at /usr/src/debug/nfs-ganesha-2.7.5-ibm056.02_asan/SAL/state_async.c:72 72 export = block->sbd_lock_entry->sle_export; (gdb) print *block $1 = {sbd_list = {next = 0x72800009, prev = 0x613000a8e128}, sbd_grant_type = STATE_GRANT_NONE, sbd_block_type = STATE_BLOCK_NONE, sbd_granted_callback = 0x0, sbd_blocked_cookie = 0x0, sbd_lock_entry = 0x0, sbd_prot = {sbd_nlm = {sbd_nlm_fh = {n_len = 0, n_bytes = 0x0}, <<<<<<lock_entry is freed, we did state_add_grant_cookie just before sbd_nlm_fh_buf = '\000' <repeats 16 times>, "\002\000\000\000\000\000 \000\000\000\341a\004Pb\000\000\000\341a\004Pb\000\000\334\341a\004Pb\000 \000\000\001b\004Pb\000\000\000\000\000\000\000\000\000"}, sbd_v4 = 0x0}} (gdb) *********************************************************************************************************************************************************************************** Crash 3: (gdb) bt #0 0x00007fd4324824fb in raise (sig=11) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:36 #1 0x0000000000443ae5 in crash_handler (signo=11, info=0x7fd00b8049f0, ctx=0x7fd00b8048c0) at /usr/src/debug/nfs-ganesha-2.7.5-ibm055.00/MainNFSD/nfs_init.c:244 #2 <signal handler called> #3 0x00000000004a3bf1 in nlm_granted_callback (obj=0x7fd36c002ba8, lock_entry=0x7fd304102d80) at /usr/src/debug/nfs-ganesha-2.7.5-ibm055.00/Protocols/NLM/nlm_util.c:609 #4 0x00000000004c217a in try_to_grant_lock (lock_entry=0x7fd304102d80) at /usr/src/debug/nfs-ganesha-2.7.5-ibm055.00/SAL/state_lock.c:1758 #5 0x00000000004c252e in process_blocked_lock_upcall (block_data=0x7fd304102320) at /usr/src/debug/nfs-ganesha-2.7.5-ibm055.00/SAL/state_lock.c:1807 #6 0x00000000004bd43f in state_blocked_lock_caller (ctx=0x7fd0400028d0) at /usr/src/debug/nfs-ganesha-2.7.5-ibm055.00/SAL/state_async.c:81 #7 0x0000000000514b92 in fridgethr_start_routine (arg=0x7fd0400028d0) at /usr/src/debug/nfs-ganesha-2.7.5-ibm055.00/support/fridgethr.c:554 #8 0x00007fd43247aea5 in start_thread (arg=0x7fd00b806700) at pthread_create.c:307 #9 0x00007fd431d818cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) f 3 #3 0x00000000004a3bf1 in nlm_granted_callback (obj=0x7fd36c002ba8, lock_entry=0x7fd304102d80) at /usr/src/debug/nfs-ganesha-2.7.5-ibm055.00/Protocols/NLM/nlm_util.c:609 609 state_nlm_client_t *nlm_grant_client = nlm_grant_owner-> so_client; The nlm_grant_owner comes from lock entry's sle_owner which is NULL. Thanks and regards, Trishali. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Trishali Nayar IBM Systems ETZ, Pune. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 0

Change in ...nfs-ganesha[next]: Set no_cleanup at miscellaneous places under state_lock
by Madhu Thorat (GerritHub) 14 Apr '20

14 Apr '20

Madhu Thorat has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489962 ) Change subject: Set no_cleanup at miscellaneous places under state_lock ...................................................................... Set no_cleanup at miscellaneous places under state_lock After taking state_lock set no_cleanup=true, to avoid abort(crash) in state_wipe_file() when put_ref() is called in following code paths: 1. state_nlm_share() -> fsal_reopen2() (in case of ESTALE) 2. state_unlock() (in case lock_list becomes empty) 3. state_deleg->conflict() -> state_deleg->conflict_impl() -> async_delegrecall() Change-Id: I857bff12497f3991f058d96d81e34c810dab6661 Signed-off-by: Madhu Thorat <madhu.punjabi(a)in.ibm.com> --- M src/SAL/state_deleg.c M src/SAL/state_lock.c M src/SAL/state_share.c 3 files changed, 13 insertions(+), 1 deletion(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/62/489962/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489962 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I857bff12497f3991f058d96d81e34c810dab6661 Gerrit-Change-Number: 489962 Gerrit-PatchSet: 1 Gerrit-Owner: Madhu Thorat <madhu.punjabi(a)in.ibm.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: Do lock_list empty check under the state lock.
by Malahal (GerritHub) 13 Apr '20

13 Apr '20

Malahal has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489939 ) Change subject: Do lock_list empty check under the state lock. ...................................................................... Do lock_list empty check under the state lock. Two threads could execute state_release_grant() at the same time and they both could find the list empty leading to both calling put_ref on the object. Have the empty list check under state lock to avoid this. Change-Id: I59f89dfed751aaa710d27857a60b497cd4c24944 Signed-off-by: Malahal Naineni <malahal(a)us.ibm.com> --- M src/SAL/state_lock.c 1 file changed, 6 insertions(+), 3 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/39/489939/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489939 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I59f89dfed751aaa710d27857a60b497cd4c24944 Gerrit-Change-Number: 489939 Gerrit-PatchSet: 1 Gerrit-Owner: Malahal <malahal(a)gmail.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: selinux: move external types to a require{} block
by Kaleb KEITHLEY (GerritHub) 10 Apr '20

10 Apr '20

Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489328 ) Change subject: selinux: move external types to a require{} block ...................................................................... selinux: move external types to a require{} block drawing inferrences from ceph's .../selinux/ceph.te it appears that these "external" types should be defined in a require block Change-Id: Ide87370535a2dff5c2ccc84bdfb1fc16cef3d0d4 Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> --- M src/selinux/ganesha.te 1 file changed, 54 insertions(+), 53 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/28/489328/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489328 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Ide87370535a2dff5c2ccc84bdfb1fc16cef3d0d4 Gerrit-Change-Number: 489328 Gerrit-PatchSet: 1 Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com> Gerrit-MessageType: newchange

1 0

Announce Push of V4-dev.12
by Frank Filz 09 Apr '20

09 Apr '20

Branch next Tag:V4-dev.12 Merge Highlights * Clean up DRC code * selinux: targeted policy additions for ceph/rgw and gluster * Breaking rename: libfsalproxy => libfsalproxy_v4. Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: f78f502 Frank S. Filz V4-dev.12 a252c6b Solomon Boulos Breaking rename: libfsalproxy => libfsalproxy_v4. 5530a5d Kaleb S. KEITHLEY selinux: additional targeted policy for gluster 167b0c2 Kaleb S. KEITHLEY selinux: targeted policy additions for ceph/rgw 878c610 Malahal Naineni Clean up DRC code

1 0

Change in ...nfs-ganesha[next]: Documenting the Security_Label option
by Arjun Sharma (GerritHub) 09 Apr '20

09 Apr '20

Arjun Sharma has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489245 ) Change subject: Documenting the Security_Label option ...................................................................... Documenting the Security_Label option Change-Id: I4d73a1571fecf455d0b8e69e924c3e8584e1a42e Signed-off-by: Arjun Sharma <arjsharm(a)redhat.com> --- M src/config_samples/config.txt M src/doc/man/ganesha-gluster-config.rst 2 files changed, 4 insertions(+), 0 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/45/489245/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489245 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I4d73a1571fecf455d0b8e69e924c3e8584e1a42e Gerrit-Change-Number: 489245 Gerrit-PatchSet: 1 Gerrit-Owner: Arjun Sharma <arjsharm(a)redhat.com> Gerrit-MessageType: newchange

1 0

ci test config
by Malahal Naineni 09 Apr '20

09 Apr '20

We found some issues while doing minimal config values, so is it possible to run tests with the following config? #1. lru_run_interval=1; #2. Entries_HWMark=1; #3. Chunks_HWMark=1; Regards, Malahal.

2 1

Change in ...nfs-ganesha[next]: selinux: additional targeted policy for gluster
by Kaleb KEITHLEY (GerritHub) 09 Apr '20

09 Apr '20

Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489237 ) Change subject: selinux: additional targeted policy for gluster ...................................................................... selinux: additional targeted policy for gluster https://bugzilla.redhat.com/show_bug.cgi?id=1822500 Change-Id: I16bf3ef8233c98ba4a44221f834d009f39118f7c Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> --- M src/selinux/ganesha.te 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/37/489237/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489237 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I16bf3ef8233c98ba4a44221f834d009f39118f7c Gerrit-Change-Number: 489237 Gerrit-PatchSet: 1 Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: selinux: targeted policy additions for ceph/rgw
by Kaleb KEITHLEY (GerritHub) 09 Apr '20

09 Apr '20

Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489233 ) Change subject: selinux: targeted policy additions for ceph/rgw ...................................................................... selinux: targeted policy additions for ceph/rgw https://bugzilla.redhat.com/show_bug.cgi?id=1821612 Change-Id: Ibed42985af31fd2984bc04e0d53011fec4b4c9be Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> --- M src/selinux/ganesha.te 1 file changed, 11 insertions(+), 0 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/33/489233/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/489233 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Ibed42985af31fd2984bc04e0d53011fec4b4c9be Gerrit-Change-Number: 489233 Gerrit-PatchSet: 1 Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com> Gerrit-MessageType: newchange

1 0