The biggest guideline to make sure you don't get corruption is to not use unstable writes. I realize this is a performance hit, but it's the only way to guarantee that every ack'd write is committed to stable storage. (Obviously, if a crash occurs mid-write, all bets are off). Ganesha does not currently have any kind if data cache. However, FSAL_VFS uses the local pagecache. A stable write and a commit will each cause an fsync(), so the result will be committed to disk. An unstable write, however, will just end up in the pagecache, and so will likely not be stable (depending, of course, on how busy the server is). Looking at the kernel client, it looks like it sends unstable writes if there are more writes queued to the file, or if a commit is already scheduled. Otherwise, it sends stable writes. Beyond that, we're outside of my knowledge of the client. I assume it will commit or perform a stable write periodically, even if writes are constant? Note that all this is unrelated to the sync/async mount flags; those determine how writes affect the client pagecache, not the server at all. Daniel On 3/12/20 2:36 AM, alxg@weka.io wrote:
Hi, We'e setting up ganesha using FSAL_VFS, and NFSv3 only. We have a cluster of them running with floating IPs being failed over between them when a host dies. The question we have is regarding writes which are UNSTABLE - meaning the client writes out from its cache for example with UNSTABLE, and then sends a COMMIT. In the event of a ganesha host crashing after receiving the writeouts, but before the COMMIT - is it assured that when the floating IP will failover to another host the client will resend the unstable writes again as well? I guess this has several parts to it: * Client side - assuming it's a linux kernel nfs client and it uses the pagecache - does it mark the pages clean right after successfully sending their writes as unstable, or does it mark them as clean only after it successfully sends a COMMIT on them as well? Do you happen to know? * Server side - Does nfs-ganesha have some data cache? Or the writes being unstable just means they can land in the pagecache of the server side, and stay there until the COMMIT does a sync? in that case do we need to set `NFS_Commit = true`?
What are the guidelines on such a setup to ensure data doesn't get lost and doesn't corrupt?
Thank you, Alex _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org