Cookies need to not change. It doesn't matter what they are, they just need to be stable. If a cookie changes, the parent directory needs to be invalidated with an invalidate UP call, with the FSAL_UP_INVALIDATE_DIR_CHUNKS and FSAL_UP_INVALIDATE_DIR_POPULATED flags set. You can just send invalidate calls and leave your setup the same as it is now, but this will result in bad directory listing performance for large directories, since the cache will be constantly invalidated. A better solution is to come up with some form of cookie that is stable in the face of a remove. Storing the dirents in some form of tree with the tree's key used as the cookie would work (that's what we do for whence-is-name FSALs). Then again, performance of such a setup can't be great, so maybe it doesn't matter? Daniel On 1/14/21 9:03 AM, Sandeep Nashikkar via Devel wrote:

Hi

We are having a samba server exporting a locally mounted ganesha share and we have our own FSAL implementation.

Samba share consistently get stuck during readdir operation on NFS export in particular sequence of operations from samba on ganesha export.

These are particular set of operations before it gets stuck:

1. There are around 160 files in the folder and readdir output looks like . .. file1 file2 file3 file4 …..and so on 2. file2 and file3 in the previous readdir output are deleted. 3. readdir_ex calls from FSAL now return following one by one . .. file1 file4 4. After it returns file4, readdir call from ganesha returns NFS4ERR_DELAY to client with following error in ganesha logs:

12/01/2021 03:24:32 : epoch 5ffc6dd5 : <machine> : ganesha.nfsd-20352[svc_168] mdcache_avl_insert_ck :NFS READDIR :DEBUG :NFS READDIR: DEBUG: Already existent when inserting dirent 0x7f0b6c0d8c40 for <filename> on entry=0x7f0b6802dfb0 FSAL cookie=4, duplicated directory cookies make READDIR unreliable.

5. ls -l  command on the directory over local mount also gets stuck for few mins. 6. Previous readdir calls logs between (before files were deleted In #2) reveal that the file4 had cookie=6 and that’s where ganesha throws an error 12/01/2021 03:24:08 : epoch 5ffc6dd5 : OneCSUnixMa.idcprodcert.loc : ganesha.nfsd-20352[svc_173] mdcache_avl_insert_ck :NFS READDIR :F_DBG :NFS READDIR: FULLDEBUG: Insert dirent 0x7f0b68020cb0 for collection-101-3540931298411264490.wt on entry=0x7f0b6802dfb0 FSAL cookie=6

So couple of questions:

1. What is the cookie value we should send to avoid this error occurring during insertion in AVL tree? We currently return the offset of the entry in cookie. 2. Do we need to invalidate the mdcache of ganesha using up-call whenever there is a change in the files inside the underlying dirent due to rename or delete operation?

Thanks

Sandeep

_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org