On Mon, 2021-07-19 at 10:37 -0400, Lars Kellogg-Stedman wrote: > On Mon, Jul 19, 2021 at 09:37:43AM -0400, Jeff Layton wrote: > > With the rados_cluster backend, you can also configure a nodeid to use. > > If you set up a trivial rados_cluster with one (active/passive) host, I > > expect it'll work more like you want. So, I'd set this in the NFSv4 > > block: > > Using `rados_cluster` and sharing a `nodeid`... > > NFSv4 > { > RecoveryBackend = rados_cluster; > Minor_Versions = 1,2; > Grace_Period = 60; > } > > [...] > > RADOS_KV > { > UserId = "nfsmeta"; > pool = "cephfs.tank.meta"; > namespace = "ganesha"; > nodeid = "nfs.storage"; > } > > ...seems to have resolved the problem; failover times now seem to > track the `Grace_Period` setting. > Excellent! Thanks for confirming it. > > Longer term, maybe we should allow rados_ng and rados_kv to set a static > > nodeid too? That would also probably fix this for people who just want a > > simple active/passive cluster. Basically we'd just need to add a > > get_nodeid operation for them. > > Was I misusing `rados_ng`? All the docs I found said "use rados_ng for > active/passive configurations". > Nope, that was an oversight on my part in implementing the RADOS_KV "nodeid" setting. At the time I added it, I was focused on rados_cluster and just didn't consider that rados_ng in an active/passive cluster would also need a similar setting. I think what we need to do is make rados_ng (and maybe rados_kv) also respect the nodeid setting. The bottom line is that when you take over serving an nfs over cephfs "service" from another entity, you need to use the same nodeid that it used before so that the MDS releases things properly. I'll see about rolling some patches and updating the docs. Thanks!