11:57 a.m.
The ganesha_mgr python script must be timing out since the DBUS service
thread is blocked in your FSAL.
I'm not sure what kind of library call you would want, and any interface
like that would be added to Ganesha V5 (we just decided this morning to
start the release candidate cycle for V4 so no more features will be
accepted into V4). V4 will have a robust dynamic export config with updates
triggered by SIGHUP though you wouldn't know when the update process is
complete. The existing interface that DBUS calls or even the interface
SIGHUP calls could be exported so you could hook up your own trigger either
by incorporating Ganesha as library into your own front end, or driving the
interface from code in your FSAL .so, either could then expose some kind of
control interface that suits your needs better. If it's just a matter of
making sure the existing interfaces are exported of course that IS something
that could drop during the release candidate cycle.
You may want to explore why the lookup_path is taking so long, no matter how
you drive the update export interfaces, a long delay in that will result in
a significant exposure where your exports are in flux.
Note that the V4 dynamic export management via SIGHUP is significantly
improved, allowing exports to be removed (if an export is no longer in the
config, it will be removed) and additional parameters including Pseudo
capable of being modified.
Frank
From: Sandeep Nashikkar via Devel [mailto:devel@lists.nfs-ganesha.org]
Sent: Tuesday, October 12, 2021 8:22 AM
To: devel(a)lists.nfs-ganesha.org
Subject: [NFS-Ganesha-Devel] ganesha_mgr add_export command returning
premature success when underlying storage is slow
Hello
We are seeing issue as following:
1. We are using our own FSAL with ganesha version 2.8.2 which
eventually performs operation on underlying storage through a wrapper layer.
2. We perform add_export programmatically when required. When
add_export command tries to lstat, sometimes the underlying storage takes
time to return the attributes. Here is the stack where it gets stuck for
some time:
#5 0x00007fdc9d24ea0f in lookup_path () from
/usr/lib64/ganesha/libfsaltdfs.so
#6 0x000000000053f719 in mdcache_lookup_path (exp_hdl=0x7fdbfa43d900,
path=0x7fdbfa40f200 "/Chk1234", handle=0x7fdbfc0daac8, attrs_out=0x0) at
/usr/src/debug/nfs-gan
esha-2.6.3/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c:1697
#7 0x0000000000504454 in init_export_root (export=0x7fdbfa435448) at
/usr/src/debug/nfs-ganesha-2.6.3/support/exports.c:2253
#8 0x00000000005029c2 in export_commit_common (node=0x7fdbfa4331a0,
link_mem=0x0, self_struct=0x7fdbfa435448, err_type=0x7fdbfc0db160,
commit_type=add_export) at /us
r/src/debug/nfs-ganesha-2.6.3/support/exports.c:1235
#9 0x0000000000502e27 in add_export_commit (node=0x7fdbfa4331a0,
link_mem=0x0, self_struct=0x7fdbfa435448, err_type=0x7fdbfc0db160) at
/usr/src/debug/nfs-ganesha-2.6
.3/support/exports.c:1351
#10 0x000000000055832b in proc_block (node=0x7fdbfa4331a0, item=0x7e51a8
<add_export_param+8>, link_mem=0x0, err_type=0x7fdbfc0db160) at
/usr/src/debug/nfs-ganesha-2.
6.3/config_parsing/config_parsing.c:1362
#11 0x0000000000559116 in load_config_from_node (tree_node=0x7fdbfa4331a0,
conf_blk=0x7e51a0 <add_export_param>, param=0x0, unique=false,
err_type=0x7fdbfc0db160) at
/usr/src/debug/nfs-ganesha-2.6.3/config_parsing/config_parsing.c:1862
#12 0x0000000000518d42 in gsh_export_addexport (args=0x7fdbfc0db220,
reply=0x7fdc97bf5180, error=0x7fdbfc0db270) at
/usr/src/debug/nfs-ganesha-2.6.3/support/export_mg
r.c:1038
#13 0x000000000055330b in dbus_message_entrypoint (conn=0x7fdc97aaa0c0,
msg=0x7fdbfa42a440, user_data=0x7e5ff0 <export_interfaces>) at
/usr/src/debug/nfs-ganesha-2.6.
3/dbus/dbus_server.c:564
#14 0x00007fdc9ca50276 in _dbus_object_tree_dispatch_and_unlock () from
/lib64/libdbus-1.so.3
#15 0x00007fdc9ca41b29 in dbus_connection_dispatch () from
/lib64/libdbus-1.so.3
#16 0x00007fdc9ca41e42 in _dbus_connection_read_write_dispatch () from
/lib64/libdbus-1.so.3
#17 0x0000000000553ed9 in gsh_dbus_thread (arg=0x0) at
/usr/src/debug/nfs-ganesha-2.6.3/dbus/dbus_server.c:796
#18 0x00007fdc9b429e25 in start_thread () from /lib64/libpthread.so.0
#19 0x00007fdc9aae6bad in clone () from /lib64/libc.so.6
3. Despite the lstat getting stuck, add_export command returns
prematurely in 30-40 seconds or so and return code is success. This is not
expected.
4. This causes subsequent mount to fail since the export is not
actually ready for the mount (init_export_root not being complete)
Can someone help here to understand why dbus interface returns success? Is
this known issue? How can we fix this behavior?
Can we rather have a library interface to create/delete/update exports
rather than relying on ganesha_mgr python script and dbus interface?
Thanks
Sandeep