Blocked Lock Not found when calling up_async_lock_grant
Hello, When reading the code, I wonder if the blocking lock can work with GPFS. When the FSAL upcalls the SAL with up_async_lock_grant, it passes handle info in gsh_buffdesc so that in lock_grant we can rebuild the handle later (using create_handle) to look for the existing blocked lock in the queue. The issue is create_handle of GPFS always creates a new pointer to obj_hdl and in the lookup fct find_blocked_lock_upcall, ganesha compares the pointer of obj_hdl and not the value of this handle, thus it will never find the existing blocked lock. Should we use a fct like handle_compare instead or did I misunderstand something? Thank you Viet
Handles are a bit confusing in Ganesha. The handle doing the pointer comparison are actually MDCACHE handles, rather than GPFS handles, and MDCACHE handles are refcounted. The SAL will keep a ref on the MDCACHE handle as long as there are any locks outstanding, so the pointer to the MDCACHE handle will be stable, and the pointer comparison should succeed. (In case you don't know about stacking FSALs in Ganesha:) Ganesha has a concept called stacking FSALs. Each export is actually a stack of FSALs, not a single one. At the bottom is the FSAL communicating with the actual filesystem, such as VFS or GPFS. At the top is MDCACHE, which provides 2 features: a stable handle cache (so upper layers can depend on pointers to handles being valid) and an (optional-but-on-by-default) metadata cache for object metadata and for directory entries. There can be other FSALs stacked in the middle, but at the moment the only other such FSAL is NULL, which does nothing but pass through, and allow testing of stacking. Each object handle belongs to a single FSAL, and has a pointer to the object handle of the FSAL below it in the stack (the sub-FSAL). This allows operations to be passed down the stack. So MDCACHE's getattrs() operation will return the attributes from it's cache, if it can, but otherwise call into the sub-FSAL's getattrs() to get the attributes. Daniel On 11/06/2018 11:45 AM, ntvietvn@gmail.com wrote:
Hello,
When reading the code, I wonder if the blocking lock can work with GPFS.
When the FSAL upcalls the SAL with up_async_lock_grant, it passes handle info in gsh_buffdesc so that in lock_grant we can rebuild the handle later (using create_handle) to look for the existing blocked lock in the queue. The issue is create_handle of GPFS always creates a new pointer to obj_hdl and in the lookup fct find_blocked_lock_upcall, ganesha compares the pointer of obj_hdl and not the value of this handle, thus it will never find the existing blocked lock.
Should we use a fct like handle_compare instead or did I misunderstand something?
Thank you Viet _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
And just to add some clarity, the create_handle that gets called is mdc_create_handle, which looks in its cache first. Assuming that nothing has caused SAL to drop its reference on the mdcache handle, it will be found and GPFS's create_handle is not called. It is through this mechanism that we avoid each FSAL having to have its own handle key to handle object hash, and why we realized mdcache must always be present. Frank
-----Original Message----- From: Daniel Gryniewicz [mailto:dang@redhat.com] Sent: Tuesday, November 6, 2018 9:57 AM To: devel@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Devel] Re: Blocked Lock Not found when calling up_async_lock_grant
Handles are a bit confusing in Ganesha. The handle doing the pointer comparison are actually MDCACHE handles, rather than GPFS handles, and MDCACHE handles are refcounted. The SAL will keep a ref on the MDCACHE handle as long as there are any locks outstanding, so the pointer to the MDCACHE handle will be stable, and the pointer comparison should succeed.
(In case you don't know about stacking FSALs in Ganesha:)
Ganesha has a concept called stacking FSALs. Each export is actually a stack of FSALs, not a single one. At the bottom is the FSAL communicating with the actual filesystem, such as VFS or GPFS. At the top is MDCACHE, which provides 2 features: a stable handle cache (so upper layers can depend on pointers to handles being valid) and an (optional-but-on-by-default) metadata cache for object metadata and for directory entries. There can be other FSALs stacked in the middle, but at the moment the only other such FSAL is NULL, which does nothing but pass through, and allow testing of stacking.
Each object handle belongs to a single FSAL, and has a pointer to the object handle of the FSAL below it in the stack (the sub-FSAL). This allows operations to be passed down the stack. So MDCACHE's getattrs() operation will return the attributes from it's cache, if it can, but otherwise call into the sub-FSAL's getattrs() to get the attributes.
Daniel
On 11/06/2018 11:45 AM, ntvietvn@gmail.com wrote:
Hello,
When reading the code, I wonder if the blocking lock can work with GPFS.
When the FSAL upcalls the SAL with up_async_lock_grant, it passes handle info in gsh_buffdesc so that in lock_grant we can rebuild the handle later (using create_handle) to look for the existing blocked lock in the queue. The issue is create_handle of GPFS always creates a new pointer to obj_hdl and in the lookup fct find_blocked_lock_upcall, ganesha compares the pointer of obj_hdl and not the value of this handle, thus it will never find the existing blocked lock.
Should we use a fct like handle_compare instead or did I misunderstand something?
Thank you Viet _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
Thank you for your clear answers ! I'll take a look at MDcache code. Viet
What version are you using? On Tue, Nov 6, 2018 at 10:16 PM <ntvietvn@gmail.com> wrote:
Hello,
When reading the code, I wonder if the blocking lock can work with GPFS.
When the FSAL upcalls the SAL with up_async_lock_grant, it passes handle info in gsh_buffdesc so that in lock_grant we can rebuild the handle later (using create_handle) to look for the existing blocked lock in the queue. The issue is create_handle of GPFS always creates a new pointer to obj_hdl and in the lookup fct find_blocked_lock_upcall, ganesha compares the pointer of obj_hdl and not the value of this handle, thus it will never find the existing blocked lock.
Should we use a fct like handle_compare instead or did I misunderstand something?
Thank you Viet _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
participants (4)
-
Daniel Gryniewicz -
Frank Filz -
Malahal Naineni -
ntvietvn＠gmail.com