[NFS-Ganesha-Support] lock issue for same clientid on the same node using NFS Ganesha 3.5

We are using NFS Ganesha 3.5 and occassionaly we see the following messages on the client host. The applications are no longer able to take locks. They are all taking RDLCKs over the entire file range. fcntl(19, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=0, l_len=1800}) The host has multiple mounts of the SAME exported filesystem. All the mounts are identical except for the mount path. Each mount is sending the the SAME SETCLIENTID to the NFS Ganesha. Any pointers much appreciated. 27/03/2024 20:28:16 : epoch 6601203a : tiger-nfs-shared-444 : ganesha.nfsd-22[svc_10] create_nfs4_owner :NFS4 LOCK :CRIT :Related {STATE_OPEN_OWNER_NFSV4 0x7f40f4041ad0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000b0ddcf9b36316) confirmed=1 seqid=1294 refcount=6} doesn't match for {STATE_LOCK_OWNER_NFSV4 0x7f40b40087f0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(20:0x6c6f636b2069643a000002e70000000000000 002) confirmed=1 seqid=2 related_owner={STATE_OPEN_OWNER_NFSV4 0x7f40c0031ed0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000a61fe2b043999) confirmed=1 seqid=1288 refcount=3} refcount=3} 27/03/2024 20:28:16 : epoch 6601203a : tiger-nfs-shared-444 : ganesha.nfsd-22[svc_10] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed to create new lock owner Lock: obj=0x7f40f001fbc8, fileid=50331779, type=READ , start=0x0, end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4 0x7f40f4041ad0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000b0ddcf9b36316) confirmed=1 seqid=1294 refcount=6} 27/03/2024 20:28:16 : epoch 6601203a : tiger-nfs-shared-444 : ganesha.nfsd-22[svc_14] create_nfs4_owner :NFS4 LOCK :CRIT :Related {STATE_OPEN_OWNER_NFSV4 0x7f40f4041ad0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000b0ddcf9b36316) confirmed=1 seqid=1302 refcount=6} doesn't match for {STATE_LOCK_OWNER_NFSV4 0x7f40b40087f0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(20:0x6c6f636b2069643a000002e70000000000000 002) confirmed=1 seqid=2 related_owner={STATE_OPEN_OWNER_NFSV4 0x7f40c0031ed0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000a61fe2b043999) confirmed=1 seqid=1288 refcount=3} refcount=3} 27/03/2024 20:28:16 : epoch 6601203a : tiger-nfs-shared-444 : ganesha.nfsd-22[svc_14] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed to create new lock owner Lock: obj=0x7f40f001fbc8, fileid=50331779, type=READ , start=0x0, end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4 0x7f40f4041ad0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000b0ddcf9b36316) confirmed=1 seqid=1302 refcount=6} 27/03/2024 20:28:16 : epoch 6601203a : tiger-nfs-shared-444 : ganesha.nfsd-22[svc_17] create_nfs4_owner :NFS4 LOCK :CRIT :Related {STATE_OPEN_OWNER_NFSV4 0x7f40f4041ad0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000b0ddcf9b36316) confirmed=1 seqid=1310 refcount=7} doesn't match for {STATE_LOCK_OWNER_NFSV4 0x7f40b40087f0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(20:0x6c6f636b2069643a000002e70000000000000 002) confirmed=1 seqid=2 related_owner={STATE_OPEN_OWNER_NFSV4 0x7f40c0031ed0: clientid={0x7f40f4004680 ClientID={Epoch=0x6601203a Counter=0x00000002} CONFIRMED Client={0x7f40f4001f40 name=(77:Linux NFSv4.0 worker007.acmelab.local/240b:c0e0:205:5500:b464:2:0:b57d) refcount=1} t_delta=0 reservations=1 refcount=7 cb_prog=1073741824 r_addr=240b:c0e0:105:5500:b463:2:3:0.135.233 r_netid=tcp6} owner=(24:0x6f70656e2069643a000002e700000000000a61fe2b043999) confirmed=1 seqid=1288 refcount=3} refcount=3}