12:40 p.m.
We are getting this crash from a couple of customers (using ganesha2.5
based code with some backports!). Looks like commit
3aa8575880190d7a20946a38ae3bc70b278d1099 (see the last paragraph of
the commit message) may fix this. Did anyone see this backtrace
before, if so what fixed it?
(gdb) bt
#0 0x00007fd511af64ab in raise () from /lib64/libpthread.so.0
#1 0x0000000000455447 in crash_handler (signo=11,
info=0x7fd47bef47f0, ctx=0x7fd47bef46c0)
at /usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/MainNFSD/nfs_init.c:225
#2 <signal handler called>
#3 0x000000000054fecf in mdcache_readdir_chunked
(directory=0x7fd1697a6c80, whence=0,
dir_state=0x7fd47bef57c0, cb=0x4330f3 <populate_dirent>,
attrmask=122830, eod_met=0x7fd47bef5e9b)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:2971
#4 0x000000000053f44c in mdcache_readdir (dir_hdl=0x7fd1697a6cb8,
whence=0x7fd47bef57a0,
dir_state=0x7fd47bef57c0, cb=0x4330f3 <populate_dirent>,
attrmask=122830, eod_met=0x7fd47bef5e9b)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c:639
#5 0x00000000004339aa in fsal_readdir (directory=0x7fd1697a6cb8,
cookie=0, nbfound=0x7fd47bef5e9c,
eod_met=0x7fd47bef5e9b, attrmask=122830, cb=0x496a1a
<nfs3_readdirplus_callback>, opaque=0x7fd47bef5e50)
at /usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/fsal_helper.c:1497
#6 0x000000000049680d in nfs3_readdirplus (arg=0x7fd1ed2071d0,
req=0x7fd1ed2069c8, res=0x7fd1589bd3d0)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/Protocols/NFS/nfs3_readdirplus.c:309
#7 0x000000000044cc86 in nfs_rpc_execute (reqdata=0x7fd1ed2069a0)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1290
#8 0x000000000044d490 in worker_run (ctx=0x4029b20)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1562
#9 0x000000000050c4b0 in fridgethr_start_routine (arg=0x4029b20)
at /usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/support/fridgethr.c:550
#10 0x00007fd511aeee25 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fd5111af34d in clone () from /lib64/libc.so.6
(gdb) frame 3
#3 0x000000000054fecf in mdcache_readdir_chunked
(directory=0x7fd1697a6c80, whence=0,
dir_state=0x7fd47bef57c0, cb=0x4330f3 <populate_dirent>,
attrmask=122830, eod_met=0x7fd47bef5e9b)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:2971
2971 if (dirent->ck == whence) {
(gdb) p dirent
$1 = (mdcache_dir_entry_t *) 0x1e58408900000000
(gdb) p dirent->ck
Cannot access memory at address 0x1e58408900000060
(gdb) l
2966 fsal_status_t status;
2967 enum fsal_dir_result cb_result;
2968 mdcache_entry_t *entry = NULL;
2969 struct attrlist attrs;
2970
2971 if (dirent->ck == whence) {
2972 /* When called with whence, the caller always wants the
2973 * next entry, skip this entry.
2974 */
2975 continue;
1:14 p.m.
That may very well fix the issue. However, that commit likely depends
on the half dozen or so commits before it, and then another set that
went in post 2.7.0, for full correctness. You may just want to extract
the refcounting bits from that patch, and see if it fixes your problem,
rather than trying to backport the entire 2.7.1 readdir to 2.5.x.
If you'd rather have the whole set, I believe I can recreate the correct
patchset from our downstream here, as that's also 2.5 based.
Daniel
On 11/06/2018 01:40 PM, Malahal Naineni wrote:
We are getting this crash from a couple of customers (using
ganesha2.5
based code with some backports!). Looks like commit
3aa8575880190d7a20946a38ae3bc70b278d1099 (see the last paragraph of
the commit message) may fix this. Did anyone see this backtrace
before, if so what fixed it?
(gdb) bt
#0 0x00007fd511af64ab in raise () from /lib64/libpthread.so.0
#1 0x0000000000455447 in crash_handler (signo=11,
info=0x7fd47bef47f0, ctx=0x7fd47bef46c0)
at /usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/MainNFSD/nfs_init.c:225
#2 <signal handler called>
#3 0x000000000054fecf in mdcache_readdir_chunked
(directory=0x7fd1697a6c80, whence=0,
dir_state=0x7fd47bef57c0, cb=0x4330f3 <populate_dirent>,
attrmask=122830, eod_met=0x7fd47bef5e9b)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:2971
#4 0x000000000053f44c in mdcache_readdir (dir_hdl=0x7fd1697a6cb8,
whence=0x7fd47bef57a0,
dir_state=0x7fd47bef57c0, cb=0x4330f3 <populate_dirent>,
attrmask=122830, eod_met=0x7fd47bef5e9b)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c:639
#5 0x00000000004339aa in fsal_readdir (directory=0x7fd1697a6cb8,
cookie=0, nbfound=0x7fd47bef5e9c,
eod_met=0x7fd47bef5e9b, attrmask=122830, cb=0x496a1a
<nfs3_readdirplus_callback>, opaque=0x7fd47bef5e50)
at /usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/fsal_helper.c:1497
#6 0x000000000049680d in nfs3_readdirplus (arg=0x7fd1ed2071d0,
req=0x7fd1ed2069c8, res=0x7fd1589bd3d0)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/Protocols/NFS/nfs3_readdirplus.c:309
#7 0x000000000044cc86 in nfs_rpc_execute (reqdata=0x7fd1ed2069a0)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1290
#8 0x000000000044d490 in worker_run (ctx=0x4029b20)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/MainNFSD/nfs_worker_thread.c:1562
#9 0x000000000050c4b0 in fridgethr_start_routine (arg=0x4029b20)
at /usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/support/fridgethr.c:550
#10 0x00007fd511aeee25 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fd5111af34d in clone () from /lib64/libc.so.6
(gdb) frame 3
#3 0x000000000054fecf in mdcache_readdir_chunked
(directory=0x7fd1697a6c80, whence=0,
dir_state=0x7fd47bef57c0, cb=0x4330f3 <populate_dirent>,
attrmask=122830, eod_met=0x7fd47bef5e9b)
at
/usr/src/debug/nfs-ganesha-2.5.3-ibm028.00-0.1.1-Source/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:2971
2971 if (dirent->ck == whence) {
(gdb) p dirent
$1 = (mdcache_dir_entry_t *) 0x1e58408900000000
(gdb) p dirent->ck
Cannot access memory at address 0x1e58408900000060
(gdb) l
2966 fsal_status_t status;
2967 enum fsal_dir_result cb_result;
2968 mdcache_entry_t *entry = NULL;
2969 struct attrlist attrs;
2970
2971 if (dirent->ck == whence) {
2972 /* When called with whence, the caller always wants the
2973 * next entry, skip this entry.
2974 */
2975 continue;
_______________________________________________
Devel mailing list -- devel(a)lists.nfs-ganesha.org
To unsubscribe send an email to devel-leave(a)lists.nfs-ganesha.org
10:27 p.m.
On Wed, Nov 7, 2018 at 12:44 AM Daniel Gryniewicz <dang(a)redhat.com> wrote:
If you'd rather have the whole set, I believe I can recreate the
correct
patchset from our downstream here, as that's also 2.5 based.
Dan, very much appreciate if you can create the whole set. We are
running into few crashes in this area.
Regards, Malahal.
9:25 a.m.
On 11/07/2018 11:27 PM, Malahal Naineni wrote:
On Wed, Nov 7, 2018 at 12:44 AM Daniel Gryniewicz
<dang(a)redhat.com> wrote:
> If you'd rather have the whole set, I believe I can recreate the correct
> patchset from our downstream here, as that's also 2.5 based.
Dan, very much appreciate if you can create the whole set. We are
running into few crashes in this area.
This should be the whole set, on top of 2.5.5:
https://github.com/dang/nfs-ganesha/tree/v2.5-readdir
Daniel
1:48 p.m.
Thank you.
On Mon, Nov 12, 2018 at 8:55 PM Daniel Gryniewicz <dang(a)redhat.com> wrote:
>
> On 11/07/2018 11:27 PM, Malahal Naineni wrote:
> > On Wed, Nov 7, 2018 at 12:44 AM Daniel Gryniewicz <dang(a)redhat.com>
wrote:
> >> If you'd rather have the whole set, I believe I can recreate the
correct
> >> patchset from our downstream here, as that's also 2.5 based.
> >
> > Dan, very much appreciate if you can create the whole set. We are
> > running into few crashes in this area.
> >
>
> This should be the whole set, on top of 2.5.5:
>
> https://github.com/dang/nfs-ganesha/tree/v2.5-readdir
>
> Daniel
>
1999
days inactive
2005
days old
4 comments
2 participants
participants (2)
-
Daniel Gryniewicz -
Malahal Naineni