6:43 a.m.
Hello everyone,
Frequent memory crashs have been occurring for few weeks in the
nfs-ganesha CEA FSAL-PROXY continuous integration test. I finally make
time for looking at these problems today by running the nfs-ganesha
server under Address Sanitizer.
I got the following stack wih a double-free error. Could anyone explain
this error ? Someone who well understand the dup-req cache ? Or someone
who already works with the code of the nfs4_op_test_stateid operation ?
The nfs4_op_test_stateid was introduce this summer by gerrit patch
418826 <https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/418826> from
fatih-acar <https://review.gerrithub.io/q/owner:fatih%2540gandi.net>;,
07/22/2018.
The dup-req cache stack seems to be involved in this error.
Regards,
Patrice
==7037==ERROR: AddressSanitizer: attempting double-free on
0x60200001ced0 in thread T7:
#0 0x480c09 in __interceptor_free (/usr/bin/ganesha.nfsd+0x480c09)
#1 0x897125 in gsh_free /opt/nfs-ganesha/src/include/abstract_mem.h:299
#2 0x896f88 in nfs4_op_test_stateid_Free
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_op_test_stateid.c:121
#3 0x703702 in nfs4_Compound_FreeOne
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_Compound.c:1081
#4 0x7042c4 in nfs4_Compound_Free
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_Compound.c:1119
#5 0x865c4a in nfs4_op_sequence
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_op_sequence.c:185
#6 0x6fd80f in nfs4_Compound
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_Compound.c:903
#7 0x67167c in nfs_rpc_process_request
/opt/nfs-ganesha/src/MainNFSD/nfs_worker_thread.c:1329
#8 0x663040 in nfs_rpc_valid_NFS
/opt/nfs-ganesha/src/MainNFSD/nfs_worker_thread.c:1539
#9 0x7ffff7bb94a1 in svc_vc_decode
/opt/nfs-ganesha/src/libntirpc/src/svc_vc.c:824
#10 0x6542ce in nfs_rpc_decode_request
/opt/nfs-ganesha/src/MainNFSD/nfs_rpc_dispatcher_thread.c:1341
#11 0x7ffff7bb934c in svc_vc_recv
/opt/nfs-ganesha/src/libntirpc/src/svc_vc.c:797
#12 0x7ffff7bb47be in svc_rqst_xprt_task
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:767
#13 0x7ffff7bb51af in svc_rqst_epoll_events
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:939
#14 0x7ffff7bb4e94 in svc_rqst_epoll_loop
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:1012:8
#15 0x7ffff7bb38bf in svc_rqst_run_task
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:1048:14
#16 0x7ffff7bc077c in work_pool_thread
/opt/nfs-ganesha/src/libntirpc/src/work_pool.c:181
#17 0x7ffff6367e24 in start_thread (/lib64/libpthread.so.0+0x7e24)
#18 0x7ffff575c34c in __clone (/lib64/libc.so.6+0xf834c)
0x60200001ced0 is located 0 bytes inside of 4-byte region
[0x60200001ced0,0x60200001ced4)
freed by thread T10 here:
#0 0x480c09 in __interceptor_free (/usr/bin/ganesha.nfsd+0x480c09)
#1 0x897125 in gsh_free /opt/nfs-ganesha/src/include/abstract_mem.h:299
#2 0x896f88 in nfs4_op_test_stateid_Free
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_op_test_stateid.c:121
#3 0x703702 in nfs4_Compound_FreeOne
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_Compound.c:1081
#4 0x7042c4 in nfs4_Compound_Free
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_Compound.c:1119
#5 0xcec2a4 in nfs_dupreq_rele
/opt/nfs-ganesha/src/RPCAL/nfs_dupreq.c:1315
#6 0x673196 in nfs_rpc_process_request
/opt/nfs-ganesha/src/MainNFSD/nfs_worker_thread.c:1442
#7 0x663040 in nfs_rpc_valid_NFS
/opt/nfs-ganesha/src/MainNFSD/nfs_worker_thread.c:1539
#8 0x7ffff7bb94a1 in svc_vc_decode
/opt/nfs-ganesha/src/libntirpc/src/svc_vc.c:824
#9 0x6542ce in nfs_rpc_decode_request
/opt/nfs-ganesha/src/MainNFSD/nfs_rpc_dispatcher_thread.c:1341
#10 0x7ffff7bb934c in svc_vc_recv
/opt/nfs-ganesha/src/libntirpc/src/svc_vc.c:797
#11 0x7ffff7bb47be in svc_rqst_xprt_task
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:767
#12 0x7ffff7bb51af in svc_rqst_epoll_events
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:939
#13 0x7ffff7bb4e94 in svc_rqst_epoll_loop
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:1012:8
#14 0x7ffff7bb38bf in svc_rqst_run_task
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:1048:14
#15 0x7ffff7bc077c in work_pool_thread
/opt/nfs-ganesha/src/libntirpc/src/work_pool.c:181
#16 0x7ffff6367e24 in start_thread (/lib64/libpthread.so.0+0x7e24)
previously allocated by thread T10 here:
#0 0x480e59 in calloc (/usr/bin/ganesha.nfsd+0x480e59)
#1 0x89689a in gsh_calloc__
/opt/nfs-ganesha/src/include/abstract_mem.h:145
#2 0x895c4e in nfs4_op_test_stateid
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_op_test_stateid.c:88:3
#3 0x6fd80f in nfs4_Compound
/opt/nfs-ganesha/src/Protocols/NFS/nfs4_Compound.c:903
#4 0x67167c in nfs_rpc_process_request
/opt/nfs-ganesha/src/MainNFSD/nfs_worker_thread.c:1329
#5 0x663040 in nfs_rpc_valid_NFS
/opt/nfs-ganesha/src/MainNFSD/nfs_worker_thread.c:1539
#6 0x7ffff7bb94a1 in svc_vc_decode
/opt/nfs-ganesha/src/libntirpc/src/svc_vc.c:824
#7 0x6542ce in nfs_rpc_decode_request
/opt/nfs-ganesha/src/MainNFSD/nfs_rpc_dispatcher_thread.c:1341
#8 0x7ffff7bb934c in svc_vc_recv
/opt/nfs-ganesha/src/libntirpc/src/svc_vc.c:797
#9 0x7ffff7bb47be in svc_rqst_xprt_task
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:767
#10 0x7ffff7bb51af in svc_rqst_epoll_events
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:939
#11 0x7ffff7bb4e94 in svc_rqst_epoll_loop
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:1012:8
#12 0x7ffff7bb38bf in svc_rqst_run_task
/opt/nfs-ganesha/src/libntirpc/src/svc_rqst.c:1048:14
#13 0x7ffff7bc077c in work_pool_thread
/opt/nfs-ganesha/src/libntirpc/src/work_pool.c:181
#14 0x7ffff6367e24 in start_thread (/lib64/libpthread.so.0+0x7e24)