11:42 p.m.
Hi — a design question rather than a bug report, and I'd like to check whether
we're leaning on behaviour that is intended before we build on it.
### Setup
- NFS-Ganesha **V6.5**, built with `USE_FSAL_VFS=ON`, and notably
`USE_RADOS_RECOV=OFF` / `USE_RADOS_GRACE=OFF`, so the rados recovery backends
are not available to us.
- One Ganesha instance per volume, running in a container, single server
(no active/active, no clustering).
- The container exports a single block-backed filesystem, and the export is the
**root** of that filesystem:
```
EXPORT {
Export_Id = 1;
Path = /data/export;
Pseudo = /;
Access_Type = RW;
Squash = No_Root_Squash;
SecType = sys;
Protocols = 4;
FSAL { Name = VFS; }
}
```
### Problem
`RecoveryBackend = fs` defaults its `RecoveryRoot` to `/var/lib/nfs/ganesha`,
which in our container is ephemeral — it is discarded on every restart, so
`clid_count` is always 0 on start-up and no client can reclaim
(`nfs4_op_open.c` returns `NFS4ERR_NO_GRACE` for `CLAIM_PREVIOUS`).
The only durable storage the server has is the exported filesystem itself, which
follows the server when it moves. So we would like `RecoveryRoot` to live on that
filesystem — but since we export its root, the recovery directory would sit
inside the exported tree, visible and writable by clients.
### What we're considering
Add a second export covering just the recovery directory, with no access:
```
EXPORT {
Export_Id = 2;
Path = /data/export/.ganesha;
Pseudo = /.ganesha;
Access_Type = None;
Protocols = 4;
FSAL { Name = VFS; }
}
```
### What we observed (V6.5, FSAL_VFS)
Both exports load:
```
Export ... 1 pseudo (/) path (/data/export) perms (... no_root_squash,
RWrw, ...)
Export ... 2 pseudo (/.ganesha) path (/data/export/.ganesha) perms (... ----, ...)
NFSv4 pseudo file system successfully initialized
```
From a client — including **root** under `No_Root_Squash`:
- `ls -a` on the share does **not** list `.ganesha` at all
- `stat`/`cat` on that path return `ENOENT`, not `EACCES`
- `rm -rf <share>/* <share>/.[!.]*` removes everything else and leaves the
recovery data untouched
- the parent export continues to serve reads and writes normally
Controlled by removing only the second export and restarting: the same directory
(same inode, mode 0700) then appears in `ls -a` and its contents are readable over
NFS. So the nested export is unambiguously what makes it unreachable.
### Questions
1. **Is this a supported use of nested exports**, or are we relying on an
accident? Is there an intended mechanism for excluding a subdirectory from an
export that we've missed?
2. **Is the READDIR filtering intentional and stable?** We expected `EACCES` on
traversal and instead the junction is omitted from the listing entirely, which
is nicer for us — but we don't want to depend on it if it's incidental.
3. **Any correctness concern with `RecoveryRoot` living inside an exported
tree?** The server writes its own client database into a path that is
nominally part of an export, albeit one no client can enter. Is there
re-entrancy, FSAL, or grace-period interaction we should worry about?
4. For a single-server, one-instance-per-volume deployment where the recovery
directory travels with the exported filesystem, **is `fs` or `fs_ng`
preferable?** We read `recovery_fs_ng.c` as adding atomic generation swaps via
`rename()` + `symlink()`; our backing filesystem is ext4, so both are viable.