Thanks, both of you. I came in with four fairly narrow questions and got
much more than that out of it, which I appreciate.
Quick update on where we landed. We've moved off 6.5 onto 15.1 and dropped
the patch we were carrying. The config that replaces it turned out to be
RecoverySkipIp rather than RecoveryBackendIPBased — I had assumed those were
two names for the same thing, and that was my own misreading rather than
anything either of you said. As far as I can tell SkipIp is what drops the
client address from the tag in nfs4_create_clid_name, while IPBased groups
the database into ip_<addr> directories keyed on cr_server_addr. Two
different halves, and I only worked that out after testing.
Martin, I did try to follow your lead on IPBased before I understood that,
so it may be worth mentioning why it didn't fit us — not a criticism of the
advice, which is clearly right for your setup. You've got a service IP that
clients follow, so whichever node takes the IP reads that IP's directory and
everyone reclaims. We don't have a floating address at all: clients reach a
per-volume ClusterIP, and the server ends up recording its own pod IP as
cr_server_addr, which changes whenever the pod is rescheduled. So IPBased
would have sharded our database by the one address guaranteed to move. We're
running it off with SkipIp on instead.
Otherwise we're staying on fs, and keeping the nested Access_Type = None
export for the recovery directory. Frank, your point about wrapping an inode
in an export ID and FSID you do have access to was the useful correction
there — we've stopped treating that export as any kind of boundary and now
describe it as simply keeping the directory out of the way. And Martin,
thanks for coming back on fs_ng; the FIXME is exactly what I was after,
rather than avoiding it on reputation alone.
If any of that reading looks off to either of you, I'd be glad to hear it —
particularly the SkipIp/IPBased split, and whether fs plus a denied nested
export is a combination you'd expect to keep working. No rush, and no need
to reply if it all sounds reasonable. Otherwise I'll stop taking up list
time. Thanks again, genuinely helpful.
cheers,
Abhinandan
On Fri, 21 Aug 2026 at 08:22, Martin Schwenke <martin(a)meltin.net> wrote:
On Thu, 20 Aug 2026 17:57:44 -0700, "Frank Filz"
<ffilzlnx(a)mindspring.com> wrote:
> How does FSID_Override prevent handle guessing?
>
> You can take the inode portion of a handle, and wrap it in export ID and
> FSID that match an export you have access to. The open_by_handle_at
system
> call will then happily access the file or directory.
Good point! That is still possible...
To secure it properly, we would have to use a subdirectory mount of the
cluster filesystem per export. With Lustre we would then need the
distinct FSID on each export to allow NFS-Ganesha to find the
"filesystem" associated with the export to get the mount_fd to do
open_by_handle_at().
I know this will be secure because resolving a handle from a subsequent
subdirectory mount doesn't work (ESTALE) if we use the mount_fd for the
first subdirectory mount... because all the subdirectory mounts
have the same default FSID. That's the behaviour we see without
FSID_Override=True.
> Good to know about recovery_fs_ng. We should take a look at that code and
> figure out if it makes any sense and either make it usable or discard it.
Yeah, as per my other reply, it wasn't a very educated guess. I took a
fairly quick look and gave up early...
peace & happiness,
martin