Re: [NFS-Ganesha-Support] HA failover in less than five minutes?
On Mon, 2021-07-19 at 10:37 -0400, Lars Kellogg-Stedman wrote:
On Mon, Jul 19, 2021 at 09:37:43AM -0400, Jeff Layton wrote:
With the rados_cluster backend, you can also configure a nodeid to use. If you set up a trivial rados_cluster with one (active/passive) host, I expect it'll work more like you want. So, I'd set this in the NFSv4 block:
Using `rados_cluster` and sharing a `nodeid`...
NFSv4 { RecoveryBackend = rados_cluster; Minor_Versions = 1,2; Grace_Period = 60; }
[...]
RADOS_KV { UserId = "nfsmeta"; pool = "cephfs.tank.meta"; namespace = "ganesha"; nodeid = "nfs.storage"; }
...seems to have resolved the problem; failover times now seem to track the `Grace_Period` setting.
Excellent! Thanks for confirming it.
Longer term, maybe we should allow rados_ng and rados_kv to set a static nodeid too? That would also probably fix this for people who just want a simple active/passive cluster. Basically we'd just need to add a get_nodeid operation for them.
Was I misusing `rados_ng`? All the docs I found said "use rados_ng for active/passive configurations".
Nope, that was an oversight on my part in implementing the RADOS_KV "nodeid" setting. At the time I added it, I was focused on rados_cluster and just didn't consider that rados_ng in an active/passive cluster would also need a similar setting. I think what we need to do is make rados_ng (and maybe rados_kv) also respect the nodeid setting. The bottom line is that when you take over serving an nfs over cephfs "service" from another entity, you need to use the same nodeid that it used before so that the MDS releases things properly. I'll see about rolling some patches and updating the docs. Thanks! -- Jeff Layton <jlayton@redhat.com>
On Tue, 2021-07-20 at 09:55 -0400, Jeff Layton wrote:
On Mon, 2021-07-19 at 10:37 -0400, Lars Kellogg-Stedman wrote:
On Mon, Jul 19, 2021 at 09:37:43AM -0400, Jeff Layton wrote:
With the rados_cluster backend, you can also configure a nodeid to use. If you set up a trivial rados_cluster with one (active/passive) host, I expect it'll work more like you want. So, I'd set this in the NFSv4 block:
Using `rados_cluster` and sharing a `nodeid`...
NFSv4 { RecoveryBackend = rados_cluster; Minor_Versions = 1,2; Grace_Period = 60; }
[...]
RADOS_KV { UserId = "nfsmeta"; pool = "cephfs.tank.meta"; namespace = "ganesha"; nodeid = "nfs.storage"; }
...seems to have resolved the problem; failover times now seem to track the `Grace_Period` setting.
Excellent! Thanks for confirming it.
Longer term, maybe we should allow rados_ng and rados_kv to set a static nodeid too? That would also probably fix this for people who just want a simple active/passive cluster. Basically we'd just need to add a get_nodeid operation for them.
Was I misusing `rados_ng`? All the docs I found said "use rados_ng for active/passive configurations".
Nope, that was an oversight on my part in implementing the RADOS_KV "nodeid" setting. At the time I added it, I was focused on rados_cluster and just didn't consider that rados_ng in an active/passive cluster would also need a similar setting.
I think what we need to do is make rados_ng (and maybe rados_kv) also respect the nodeid setting. The bottom line is that when you take over serving an nfs over cephfs "service" from another entity, you need to use the same nodeid that it used before so that the MDS releases things properly.
I'll see about rolling some patches and updating the docs.
Thanks!
It was pretty simple to do, though I haven't done any real testing with it so far. If you feel inclined to try it out, here's the patch: https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/521021 With that you should be able to use rados_ng, as long as you ensure that you set the same nodeid on both hosts. That said, there is nothing wrong with using rados_cluster in this configuration either, and that should allow you to eventually scale-out to multiple active nodes seamlessly if you choose. I sort of wonder if we ought to phase-out rados_ng altogether and just clearly document how to use rados_cluster in an active/passive configuration. -- Jeff Layton <jlayton@redhat.com>
participants (1)
-
Jeff Layton