owner parameter in lock_op2
Hi, I'm implementing a file system FSAL which is similar to CEPH FSAL. I checked ceph_fsal_lock_op2() and other FSAL lock_op2 implementation. The owner parameter is a pointer, but it converts it to 64bit integer and sends to CEPH. Ceph will judge if it's a conflict according to this value. My question is : 1. it's possible that different clients have the same value even though the possibility is very low. 2. When this sever reboot and reclaim the lock, this value should change. Thanks, Marvin
Since FSAL_CEPH is just using the pointer to the state_owner_t as a 64-bit integer, it cannot be duplicated. That pointer is unique. You can use the NFS4 or NLM owner if you want (it's stored in the state_owner_t). Those are unique, because they're based on the client ID, which is guaranteed unique to a client. In the case of FSAL_CEPH, the pointer will have changed across reboot. I'm not sure how it's handling it. Jeff would know more. Daniel On 5/8/19 3:29 AM, fanzi2009@hotmail.com wrote:
Hi, I'm implementing a file system FSAL which is similar to CEPH FSAL. I checked ceph_fsal_lock_op2() and other FSAL lock_op2 implementation. The owner parameter is a pointer, but it converts it to 64bit integer and sends to CEPH. Ceph will judge if it's a conflict according to this value. My question is : 1. it's possible that different clients have the same value even though the possibility is very low. 2. When this sever reboot and reclaim the lock, this value should change.
Thanks, Marvin _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
Since FSAL_CEPH is just using the pointer to the state_owner_t as a 64-bit integer, it cannot be duplicated. That pointer is unique. You can use the NFS4 or NLM owner if you want (it's stored in the state_owner_t). Those are unique, because they're based on the client ID, which is guaranteed unique to a client.
In the case of FSAL_CEPH, the pointer will have changed across reboot. I'm not sure how it's handling it. Jeff would know more.
To be clear, the NLM lock owner is a client generated opaque string. There is no guarantee that it is unique across all clients. I'm pretty sure the NFS4 owner is the same way. The pointer is unique within an instance of Ganesha because it is the pointer to the state_owner_t that describes a specific client lock owner, which is indexed by the possibly/probably non-unique lock owner opaque from the client PLUS the client's IP address (for NLM) and clientid (for NFS4). A clustered filesystem would need to add its own "client" or "node" identifier. HA failover traditionally relies on use of the NLM and NFSv4 state (lock and open) recovery mechanism to re-acquire locks from the failed node. With this, the recovery process need not know how the lock owners from the failed node are identified. Jeff Layton added functionality into cephfs to prevent conflicting state from being granted after a cephfs client fails until the Ganesha clients establish grace period. In some HA failover situations, it may be possible for the new node to indicate it is recovering a lock, and reclaim a lock that is still instantiated by the failed node (in this case, the failed node must not actually be failed, but perhaps is considered down because a client side network interface is down, or it could be a failback situation where the IP address for a failed node is being returned to that node after it recovers). FSAL_GLUSTER's clustering implementation technically leaves a window where a node has failed and taken it's lock state with it before Ganesha recognizes the loss of node and triggers grace period. During this window, a client could claim a conflicting lock from another node. Jeff can go into all of what he did for cephfs if you are interested. Frank
Hi, I'm implementing a file system FSAL which is similar to CEPH FSAL. I checked ceph_fsal_lock_op2() and other FSAL lock_op2 implementation. The owner
My question is : 1. it's possible that different clients have the same value even though the
On 5/8/19 3:29 AM, fanzi2009@hotmail.com wrote: parameter is a pointer, but it converts it to 64bit integer and sends to CEPH. Ceph will judge if it's a conflict according to this value. possibility is very low.
2. When this sever reboot and reclaim the lock, this value should change.
Thanks, Marvin _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
On 5/8/19 10:29 AM, Frank Filz wrote:
Since FSAL_CEPH is just using the pointer to the state_owner_t as a 64-bit integer, it cannot be duplicated. That pointer is unique. You can use the NFS4 or NLM owner if you want (it's stored in the state_owner_t). Those are unique, because they're based on the client ID, which is guaranteed unique to a client.
In the case of FSAL_CEPH, the pointer will have changed across reboot. I'm not sure how it's handling it. Jeff would know more.
To be clear, the NLM lock owner is a client generated opaque string. There is no guarantee that it is unique across all clients. I'm pretty sure the NFS4 owner is the same way.
This is true, but our state_owner_t (which is what is passed in) already has a client ID in it, so the FSAL can just use that to make it unique. That's what I meant. Daniel
I'm actually not that familiar with cephfs filelocking code, but I believe the "owner" value that we pass down to the client is only significant to that particular MDS client session. When the server reboots and reclaims the lock, that value will change, but that's ok, as that new value will also be unique and only significant within the context of the new CephFS MDS client session. On Wed, 2019-05-08 at 08:46 -0400, Daniel Gryniewicz wrote:
Since FSAL_CEPH is just using the pointer to the state_owner_t as a 64-bit integer, it cannot be duplicated. That pointer is unique. You can use the NFS4 or NLM owner if you want (it's stored in the state_owner_t). Those are unique, because they're based on the client ID, which is guaranteed unique to a client.
In the case of FSAL_CEPH, the pointer will have changed across reboot. I'm not sure how it's handling it. Jeff would know more.
Daniel
On 5/8/19 3:29 AM, fanzi2009@hotmail.com wrote:
Hi, I'm implementing a file system FSAL which is similar to CEPH FSAL. I checked ceph_fsal_lock_op2() and other FSAL lock_op2 implementation. The owner parameter is a pointer, but it converts it to 64bit integer and sends to CEPH. Ceph will judge if it's a conflict according to this value. My question is : 1. it's possible that different clients have the same value even though the possibility is very low. 2. When this sever reboot and reclaim the lock, this value should change.
Thanks, Marvin _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
-- Jeff Layton <jlayton@poochiereds.net>
participants (4)
-
Daniel Gryniewicz -
fanzi2009＠hotmail.com -
Frank Filz -
Jeff Layton